howto-build.txt

Additional libraries/packages/tools needed for Fail*:

	libprotobuf-dev
	libpthread
	libpcl1-dev
	libboost-dev
	libboost-all-dev (or at least libboost-thread-dev)
	protobuf-compiler

	cmake
	cmake-curses-gui

	AspectC++ (ag++, ac++)
	  (AspectC++ 1.1 or newer is known to work and can be obtained from
	   http://www.aspectc.org ; nightlies can be downloaded from
           http://akut.aspectc.org)

For distribution/parallelization:
	rsync
	tmux

32-bit FailBochs on x86_64 Linux machines:
	libc6-i386
	+ all libraries listed by "ldd bochs|awk '{print $3}'" in ~/bochslibs
	(client.sh will add these to LD_LIBRARY_PATH)

===========
Build steps
===========

Build Fail*
===========
For the first time
------------------
cd failbochs/

> (optional) Cleanup previous cmake remnants
find -name CMakeCache.txt | xargs rm

> Create out of source build directory
mkdir build

> Enter out-of-source build directory
> All generated files end up there.
cd build

>> Generate cmake environment. 
cmake ..
>> Setup build configuration
> Select BUILD_BOCHS or BUILD_OVP
> Select an experiment to enable by naming its experiments/ subdirectory under
>   EXPERIMENTS_ACTIVATED
> Configure Fail* features you need for this experiment by enabling CONFIG_*
> options.
> Press 'c', 'g' to regenerate the build system
> (alternatively use "cmake-gui ." for a Qt GUI)
ccmake .

> Additionally make sure Bochs is at least configured (see below).

After changes to Fail* code
---------------------------
> Prerequisite, if you're building with Bochs: configure Bochs (see below).
>
> Compile (optionally add -jN for parallel building)
make

> cmake will build all Fail* libraries, merge them into a libfail.a and put it
> into failbochs/fail.  (as the current Bochs Makefile expects it there..)

> You may use scripts/rebuild-bochs.sh to speed up repetitive tasks regarding
> Fail/Bochs builds.  This script contains a concise documentation on itself.

Doxygen documentation
---------------------
make doc
firefox core/doc/html/index.html

Debug build
-----------
> Choose "Debug" as the build type:
ccmake .

Add new (fail*) sources to build chain
--------------------------------------
To add new files to the build, see CMakeLists.txt in fail/core, and probably
consultate the CMake docs: http://cmake.org/cmake/help/documentation.html

Add new user-defined experiment/campaign
----------------------------------------
Look at experiments/coolchecksum/ as a template.  After creating a new
subdirectory in experiments/, activate it in the cmake configuration step (see
above).


Build Bochs
===========
For the first time
------------------
cd ../simulators/bochs
> Sufficient:
./configure --prefix=$(echo ~/localroot/usr) --enable-{cpu-level=6,ne2000,trace-cache,gdb-stub} --disable-docbook
> More optimised:
./configure --prefix=$(echo ~/localroot/usr) --enable-{a20-pin,x86-64,cpu-level=6,ne2000,acpi,pci,usb,repeat-speedups,trace-cache,fast-function-calls,host-specific-asms,disasm,all-optimizations,readline,clgd54xx,fpu,vmx=2,monitor-mwait,cdrom,sb16=linux,gdb-stub} --disable-docbook --with-all-libs
> Instead of --with-all-libs, you could use --with-nogui for "headless"
> experiments, additionally removing lots of library dependencies (thus
> reducing startup overhead).  --with-x11 enables only the "x" (X11), --with-wx
> only the "wx" (wxWidgets) GUI.  Note that "wx" does not play well together
> with Fail*'s "restore" feature (FailBochs will fall back to "x" if available,
> or die trying.)
>
> Once you know everything works as it should, you may want to add the
> following flags before running larger campaigns:
> --disable-logging --disable-assert-checks
>
> FIXME remove more redundant flags/libraries

After changes to Bochs code or Bochs-affecting aspects
------------------------------------------------------
> The make call from the make-ag++.sh is now invokable by calling (still in
> your build dir, optionally adding -jN for parallel building):
cd ../build
make bochs

> The former make all-clean is now invokable by
make bochsallclean

> For installing the bochs executable (former make install)
make bochsinstall
> (see >make help< for a target listing.)

> You may use scripts/rebuild-bochs.sh to speed up repetitive tasks regarding
> Fail/Bochs builds.  This script contains a concise documentation on itself.

Debug build
-----------
> Configure Bochs to use debugging-related compiler flags:
cd ../bochs
CFLAGS="-g -O0" CXXFLAGS="-g -O0" ./configure --prefix=... ... (see above)

Profiling-based optimization build
----------------------------------
> FIXME: ag++ needs to be run with --keep_woven
> Configure Bochs to use compiler flags to enable profiling:
cd ../bochs
CFLAGS="-fprofile-generate" CXXFLAGS="-fprofile-generate" LDFLAGS="-fprofile-generate" ./configure --prefix=... ... (see above)
> Build Bochs normally, and run it.
> Configure Bochs to use compiler flags to enable optimizations based on profiling results:
CFLAGS="-fprofile-use -Wcoverage-mismatch" CXXFLAGS="-fprofile-use -Wcoverage-mismatch" LDFLAGS="-fprofile-use" ./configure --prefix=... ... (see above)

Benchmarking
------------
> Simple instructions/second measurement:
> - configure Bochs with --enable-show-ips (see above)
> - bochsrc: print_timestamps: enabled=1
> Comparison IPS numbers are shown in the default bochsrc.
> Headless bochsrc configuration, all aspects disabled, guest system executes
> endless loop, host CPU Xeon X5470 (3.33GHz): IPS: 66124283 average = 66964789

Steps to run a boot image in FailBochs:
---------------------------------------
Follow the Bochs documentation, and start your own "bochsrc" configuration file based on the
$PREFIX/share/doc/bochs/bochsrc-sample.txt template (or
/usr/share/doc/bochs/examples/bochsrc.gz on Debian systems with Bochs
installed).
 -  Add your floppy/cdrom/hdd image in the floppya/ata0-master/ata0-slave
    sections; configure the boot: section appropriately.
 -  Comment out com1 and parport1.
 -  For "headless" experiments:
    config_interface: textconfig
    display_library: nogui
 -  For an X11 GUI:
    config_interface: textconfig
    display_library: x
 -  For a wxWidgets GUI (does not play well with Fail*'s "restore" feature):
    config_interface: wx
    display_library: wx
 -  Reduce the guest system's RAM to a minimum to reduce FailBochs's memory
    footprint and save/restore overhead, e.g.:
    memory: guest=16, host=16
 -  If you want to redirect FailBochs's output to a file using the shell's
    redirection operator '>', make sure /dev/stdout is not used as a target
    file for logging.  (The Debian bochsrc template unfortunately does this in
    two places.  It suffices to comment out these entries.)
 -  To make FailBochs terminate if something unexpected happens in a larger
    campaign, be sure it doesn't "ask" in these cases, e.g.:
    panic: action=fatal
    error: action=fatal
    info: action=ignore
    debug: action=ignore
    pass: action=ignore
 -  If you need a quick-and-dirty way to pass data from the guest system to the
    outside world, and you don't want to write an experiment utilizing
    GuestEvents, you can use the "port e9 hack" that prints all outb's to port
    0xe9 to the console:
    port_e9_hack: enabled=1
 -  Determinism:  (Fail)Bochs is deterministic regarding timer interrupts,
    i.e., two experiment runs after calling simulator.restore() will count the
    same number of instructions between two interrupts.  Though, you need to be
    careful when running (Fail)Bochs with a GUI enabled:  Typing "bochs -q<return>"
    on the command line may lead to the GUI window receiving a "return key
    released" event, resulting in a keyboard interrupt for the guest system.
    This can be avoided by starting Bochs with "sleep 1; bochs -q", or
    disabling the GUI (see "headless experiments" above).


Build OVP for Cortex-M3
=======================
For the first time
------------------
> Get a license from ovpworld.org
> Download:
>  - Downloads -> Main OVP Download including OVPsim Simulator
>  - Downloads -> ARM -> ARM OVP Cortex-M Profile Model
>  -                  -> Self contained ARM Cortex-M examples
>  -                  -> ARM GNU GCC and GDB tools
> Install OVP by running the self-extracting archives.
>
> Get Sourcery CodeBench Lite Edition from
> http://www.mentor.com/embedded-software/codesourcery
>  - ARM processors -> EABI release
> Install the self-extracting archive, or use the installation in
> /fs/proj/compiler/codesourcery-arm-eabi (DO, ios/kos)
>
> TODO source setup.sh, setupImperas ...
> TODO hard-coded paths

=====================================
Example experiments and code snippets
=====================================

hscsimple
---------
A simple standalone experiment (without a separate campaign).

1. Add "hscsimple" to ccmake's EXPERIMENTS_ACTIVATED.
2. Enable CONFIG_EVENT_BREAKPOINTS, CONFIG_SR_RESTORE and CONFIG_SR_SAVE.
3. Build Fail* and Bochs as described.
4. Enter experiment_targets/hscsimple/, bunzip2 -k *.bz2
5. Run "bochs -q".  After successfully booting the eCos/hello world example,
   the console shows "[HSC] breakpoint reached, saving", and a hello.state/
   subdirectory appears.
   - You probably need to adjust the bochsrc's paths to romimage/vgaromimage.
     These by default point to the locations installed by the Debian packages
     "bochsbios" and "vgabios"; for example, you alternatively may use the
   BIOSes supplied in fail/bochs/bios/.
6. Compile the experiment's second step: edit
   fail/core/experiments/hscsimple/experiment.cc, and change the first "#if 1"
   into "#if 0".  Make an incremental build, e.g., by running
   "fail/scripts/rebuild-bochs.sh -".
7. Back to experiment_targets/hscsimple/, run "bochs -q".  After restoring the
   state, the hello world program's calculation should yield a different
   result.

coolchecksum
------------
An example for separate campaign/experiment implementations.

1. Run step #1 (and if you're curious how COOL_ECC_NUMINSTR in
   experimentInfo.hpp was figured out, then step #2) of the experiment
   (analogous to what needed to be done in case of the hscsimple experiment,
   see above).  The experiment's target guest system can be found under
   experiment_targets/coolchecksum/.
   (If you want to enable COOL_FAULTSPACE_PRUNING, step #2 is mandatory because
   it generates the instruction/memory access trace needed for pruning.)
2. Build the campaign server: make coolchecksum-server
3. Run the campaign server: bin/coolchecksum-server
4. In another terminal, run step #3 of the experiment ("bochs -q").

Step #3 of the experiment currently runs 2000 experiment iterations and then
terminates, because Bochs has some memory leak issues.  You need to re-run
Bochs for the next 2k experiments.

The experiments can be significantly sped up by a) parallelization (run more
FailBochs clients; TODO document fail/scripts/* stuff) and b) a headless (and
more optimized) FailBochs configuration (see above).

MHTestCampaign
--------------
An example for separate campaign/experiment implementations.

1. execute Campaign (job server):
 <your_build_dir>/bin/MHTestCampaign-server
2. run the FailBochs instance, in properly defined environment:
 bochs

Dwarf example (tests/dwarf.cc)
------------------------------
Dependencies: 
	LibDwarf: (debian/ubuntu package: libdwarf-dev)
	LibElf:	  (debian/ubuntu package: libelf-dev)

Build with cmake:
cd {your-out-of-source-dir}
cmake {path to fail dir}
	-> cmake checks if libelf and libdwarf are present

make dwarf
./tests/dwarf main.elf

===============
Parallelization
===============
FIXME more details

Shell scripts supporting experiment distribution/parallelization
----------------------------------------------------------------
These can be found in fail/scripts/ (for now have a look at the script files
themselves, they contain some documentation):
 -  fail-env.sh -- Environment variables for distribution/parallelization host
    lists etc.; don't modify in-place but edit your own copy!
 -  distribute-experiment.sh -- Distribute necessary FailBochs ingredients to
    experiment hosts
 -  runcampaign.sh -- Locally run a campaign server, and a large amount of
    clients on the experiment hosts
 -  multiple-clients.sh -- Is run on an experiment host by runcampaign.sh,
    starts several instances of client.sh in a tmux session
 -  client.sh -- (Repeatedly) runs a single FailBochs instance

Some useful things to note:
---------------------------
 -  Using the distribute-experiment.sh script causes the local bochs binary to
    be copied to the hosts. If the binary is not present in the current directory
    the default bochs binary (-> which bochs) will be used. If you have modified
    some of your experiment code (id est, your bochs binary will change), don't
    forget to delete the local bochs binary in order to distribute the *new* binary.
 -  The runcampaign.sh script prints some status information about the clients
    recently started. In addition, there will be a few error messages concerning
    ssh, tmux and so on. They can be ignored for now.
 -  The runcampaign.sh script starts the coolchecksum-server. Note that the server
    instance will terminate immediatly (without notice), if there is still an
    existing coolcampaign.csv file.