Debugging toolset

When encountering bugs — either a segmentation fault, or some algorithm does not operate as expected — some of the tools described in this section might be helpful in figuring out the source of error.

Testcase minimizer

Header: mockturtle/algorithms/testcase_minimizer.hpp

Often, the testcase that triggers the bug is big. The testcase minimizer helps reducing the size of the testcase as much as possible, while keeping the buggy behavior triggered. A minimized testcase not only facilitates debugging, but also enhances communication between developers if the original testcase cannot be disclosed.

struct testcase_minimizer_params

Parameters for testcase_minimizer.

Public Types

enum [anonymous]

File format of the testcase.

Values:

enumerator verilog

enumerator aiger

Public Members

enum mockturtle::testcase_minimizer_params::[anonymous] file_format = verilog : File format of the testcase.

std::string path = {"."}: Path to find the initial test case and to store the minimized test case.

std::string init_case = {"testcase"}: File name of the initial test case (excluding extension).

std::string minimized_case = {"minimized"}: File name of the minimized test case (excluding extension).

std::optional<uint32_t> num_iterations = {std::nullopt}: Maximum number of iterations in total. nullopt = infinity.

std::optional<uint32_t> num_iterations_stage = {std::nullopt}: Step into the next stage if nothing works for this number of iterations.

bool verbose = {false}: Be verbose.

uint64_t seed = {0xcafeaffe}: Seed of the random generator.

template<typename Ntk> class testcase_minimizer

Debugging testcase minimizer.

Given a (sequence of) algorithm(s) and a testcase that is known to trigger a bug in the algorithm(s), this utility minimizes the testcase by trying to remove parts of the network with an increasing granularity in each stage. Only changes after which the bug is still triggered are kept; otherwise, the change is reverted.

The script of algorithm(s) to be run can be provided as (1) a lambda function taking a network as input and returning a Boolean, which is true if the script runs normally and is false otherwise (i.e. the buggy behavior is observed); or (2) (not supported on Windows platform) a lambda function making a command string to be called, taking a filename string as input. The command should return 0 if the program runs normally, return 1 if the concerned buggy behavior is observed, and return other values if the input network is not valid (thus the latest change will not be kept). If the command segfaults or an assertion fails, it is treated as observing the buggy behavior.

Usage

auto opt = []( mig_network ntk ) -> bool {
  direct_resynthesis<mig_network> resyn;
  refactoring( ntk, resyn );
  return network_is_acyclic( ntk );
};

auto make_command = []( std::string const& filename ) -> std::string {
  return "./abc -c \"read " + filename + "; drw\"";
};

testcase_minimizer_params ps;
ps.path = "."; // current directory
testcase_minimizer<mig_network> minimizer( ps );
minimizer.run( opt ); // debug algorithms implemented in mockturtle
minimizer.run( make_command ); // debug external scripts

Fuzz testing

Header: mockturtle/algorithms/network_fuzz_tester.hpp

If minimizing the testcase is not successful, is too slow, or there is not an initial testcase to start with, fuzz testing can help generating small testcases that triggers unwanted behaviors.

struct fuzz_tester_params

Parameters for testcase_minimizer.

Public Types

enum [anonymous]

File format to be generated.

Values:

enumerator verilog

enumerator aiger

Public Members

enum mockturtle::fuzz_tester_params::[anonymous] file_format = verilog : File format to be generated.

std::string filename = {"fuzz_test.v"}: Name of the generated testcase file.

std::optional<std::string> outputfile = {std::nullopt}: Filename written out by the command (to do CEC with the input testcase).

std::optional<uint64_t> num_iterations = {std::nullopt}: Max number of networks to test: nullopt means infinity.

std::optional<uint64_t> timeout = {std::nullopt}: Timeout in seconds: nullopt means infinity.

template<typename Ntk, class NetworkGenerator> class network_fuzz_tester

Network fuzz tester.

Runs an algorithm on many small random logic networks. Fuzz testing is often useful to detect potential segmentation faults in new implementations. The generated benchmarks are saved first in a file. If a segmentation fault or unexpected behavior occurs, the file can be used to reproduce and debug the problem.

The entry function run generates different networks with the same number of PIs and gates. The function run_incremental, on the other hand, generates networks of increasing sizes. These functions return true if it was terminated by an unexpected behavior, or return false if it terminates normally after the specified number of iterations without observing any defect.

The script of algorithm(s) to be tested can be provided as (1) a lambda function taking a network as input and returning a Boolean, which is true if the algorithm behaves as expected; or (2) a lambda function making a command string to be called, taking a filename string as input (not supported on Windows platform). If the command exits normally (with return value 0), CEC will be performed on the output file; otherwise (segfault, assertion fail, or return value is not 0), the fuzzer is terminated.

Usage

#include <mockturtle/algorithms/aig_resub.hpp>
#include <mockturtle/algorithms/cleanup.hpp>
#include <mockturtle/algorithms/network_fuzz_tester.hpp>
#include <mockturtle/algorithms/resubstitution.hpp>
#include <mockturtle/generators/random_logic_generator.hpp>

auto opt = [&]( aig_network aig ) -> bool {
  resubstitution_params ps;
  resubstitution_stats st;
  aig_resubstitution( aig, ps, &st );
  aig = cleanup_dangling( aig );
  return true;
};

fuzz_tester_params ps;
ps.num_iterations = 100;
auto gen = default_random_aig_generator();
network_fuzz_tester fuzzer( gen, ps );
fuzzer.run( opt );

Debugging utilities

Header: mockturtle/utils/debugging_utils.hpp

Some utility functions are provided in this header file. They can be added as assertions in algorithms to identify abnormal network operations, or be used as the checks in testcase minimizer or fuzz testing.

template<typename Ntk> bool mockturtle::network_is_acyclic(Ntk const &ntk)

Check if a network is acyclic.

This utility function checks if the network is acyclic, i.e., there is no path from a node to itself.

This function requires paint of the network (provided by wrapping with color_view).

template<typename Ntk> inline uint64_t mockturtle::count_dead_nodes(Ntk const &ntk)

Counts dead nodes in a network.

This utility function counts how many nodes in the network are said to be dead (i.e., is_dead returns true).

template<typename Ntk> inline uint64_t mockturtle::count_dangling_roots(Ntk const &ntk)

Counts dangling roots in a network.

This utility function counts how many nodes in the network have a fanout size of zero. Note that it does not skip the nodes which are marked as dead, which are normally skipped when using foreach functions.

template<typename Ntk> inline uint64_t mockturtle::count_reachable_dead_nodes(Ntk const &ntk)

Counts reachable dead nodes in a network.

This utility function counts how many nodes in the network are said to be dead (i.e., is_dead returns true) and are reachable from an output.

This function requires the paint by the network (provided by wrapping with color_view).

template<typename Ntk> inline uint64_t mockturtle::count_reachable_dead_nodes_from_node(Ntk const &ntk, typename Ntk::node const &n)

Counts dead nodes that are reachable from a given node.

This utility function counts how many nodes in the network are said to be dead (i.e., is_dead returns true) and are reachable from a given node (i.e., in the transitive fanin cone of this node).

This function requires paint of the network (provided by wrapping with color_view).

template<typename Ntk> uint64_t mockturtle::count_nodes_with_dead_fanins(Ntk const &ntk)

Counts nodes with dead fanin(s) in a network.

This utility function counts how many (not-dead) nodes in the network have at least one fanin being dead.

template<typename Ntk> bool mockturtle::check_network_levels(Ntk const &ntk)

Check the level information of a network.

This utility function checks if the levels of each node in the network and the depth of the network are correct.

This function requires level of the network (provided by wrapping with depth_view).

template<typename Ntk> bool mockturtle::check_fanouts(Ntk const &ntk)

Check the fanout information of a network.

This utility function checks if the fanouts of each node in the network are correct.

This function requires foreach_fanout of the network (provided by wrapping with fanout_view).

template<typename Ntk, typename NtkWin> bool mockturtle::check_window_equivalence(Ntk const &ntk, std::vector<typename Ntk::node> const &inputs, std::vector<typename Ntk::signal> const &outputs, std::vector<typename Ntk::node> const &gates, NtkWin const &win_opt)

Check functional equivalence between a window in a network and a stand-alone window.

This utility function checks if a window in a network, defined by a set of inputs, a set of gates (internal nodes), and a set of outputs, is functionally equivalent to an extracted window represented as a stand-alone network.

Visualization

Drawing a figure

When the testcase is small enough, it becomes possible to visualize the network. mockturtle supports writing out a network into the DOT format, which can then be visualized using Graphviz. (Write into DOT files (Graphviz))

Printing method

Header: mockturtle/utils/debugging_utils.hpp

template<typename Ntk> inline void mockturtle::print(Ntk const &ntk)

Prints information of all nodes in a network.

This utility function prints the following information for all nodes in the network:

ID
Fanin signals, if any
Level, if level is provided for the network type
Whether the node is dead
Reference count (fanout size)
Visited marker
Custom value data

It also prints the outputs of the network.

Time machine

Coming soon…