disasm-test

The purpose of this program is to ensure that the disassembly as
generated by s390_disasm matches what objdump -d produces for any
given insn. As such the program runs as part of "make regtest".


How it works
------------
1) Given an opcode, generate a C file with testcases.
2) Compile the C file.
3) Run objdump -d on the object file and capture the result in a file.
   This file will be referred to as "the objdump file".
4) Read the objdump file, extract the insn bytes and disassembly.
5) Feed the so-extracted insn bytes into VEX's decode-and-irgen
   machinery with disassembly (= tracing frontend) being enabled.
6) Intercept the so-disassembled text and compare it with the
   disassembly in the objdump file.


Invocation
----------
See disasm-test --help for the most up-to-date instructions.

disasm-test --all
  For all opcodes known to disasm-test, generate testcases and
  verify the disassembly. This is how disasm-test is invoked
  during regression testing.

disasm-test --generate OPCODE_NAMEs
  For each specified opcode, generate a C file with testcases,
  compile it and create the objdump file.
  Useful when adding new opcodes.

disasm-test --verify OBJDUMP_FILEs
  For each specified objdump file, verify that the disassembly via VEX
  matches. Useful when adding new opcodes.

disasm-test --run OPCODE_NAMEs
  Combines --generate and --verify. Useful when adding new opcodes.


Other non-debugging options
---------------------------
--verbose
  Reports activity and miscellaneous information deemed interesting.

--summary
  Write out test generation summary. This option is only observed in
  combination with --all.

--gcc=/path/to/gcc
  Specify which GCC to use. Default is: gcc on $PATH.

--gcc-flags=FLAGS
  Specify which flags GCC to use. Default is: "-march=arch14".

--objdump=/path/to/objdump
  Specify which objdump to use. Default is: objdump on $PATH.

--keep-temp
  Keep generated files: .c file with testcases, object file, objdump
  file, and .vex file

--show-exc-spec
  Show generated insns that cause specification exceptions.
  Because insns causing specification exceptions are typically accepted
  by GCC and objdump an objdump file may contain them. But comparing
  them is pointless.

--no-show-miscompares
  Do not show miscomparing insns.


Debugging options
-----------------
--debug
  Additional debugging output.


Testcase generation
-------------------
Testcase generation is not exhaustive. That would produce too large a
number of testcases. For example, testing CRB R1,R2,M3,D4(B4)
exhaustively would produce 16x16x16x12x16 = 786432 testcases. Instead,
we attempt to create "interesting" testcases. Like so:

- for an AR, FPR or VR operand pick a register at random
- for a GPR operand that is not an index or base register pick a GPR
  at random
- for a GPR that is base or index register, pick r0 and another GPR
  at random
- for a 12-bit displacement pick 0, 1, 2, 4095
- for a 20-bit displacement pick -524288, -2, -1, 0, 1, 2, 524287
- for a signed integer field pick the boundary values of its domain and
  -2,-1,0,1,2
- for an unsigned integer field pick the boundary values of its domain
  and 1, 2
- for a mask field, pick all allowed values

Why are we picking these values? Boundary cases are *always*
interesting, as is 0. '1' is picked because it is odd and '2' is picked
because it is even.

Note: if an opcode has more than one GPR (without specification) choose
different registers. We do this because we want to catch bugs due to
mixed up registers.
E.g. If the testcase is "brxh r1,r2,8" and s390_disasm produces
"brxh r2,r1,8" we want to catch that. Obviously, to do so the registers
need to be different. The same applies to ARs, FPRs and VRs.


Adding a new opcode
-------------------
See extensive documentation at the top of file opcode.c
Any opcode can be added. It is not necessary for the opcode to have
extended mnemonics.


Integration into regression testing
-----------------------------------
1) Observe the exit code

   disasm-test --all --no-show-miscompares

   There will be no output to stdout and stderr. If there are no
   miscompares in the disassembly the exit code will be 0. Otherwise,
   it will be 1.

2) Observe stderr

   disasm-test --all

   Miscomparing disassembly will be written to stderr. Exit code as
   described above.

3) Observe testcase summary

   disasm-test --all --no-show-miscompares --summary

   Will write information about #testcases as well as failing ones
   to stdout. Exit code as described above.


Status
------
All opcodes which are implemented in valgrind are considered.


TODO
----
(1) Testcase generation for the "Rotate and ..." family of opcodes needs
    to be improved. Several interesting cases involving the T- and Z-bit
    are not considered.

(2) Different versions of objdump produce slightly different disassembly.
    In 2.38 base register 0 is written as "%r0". In current binutils git
    that register will be output as "0".

(3) Generated testcases may cause specification exceptions. This
    happens iff a constraint involves more than one opcode field.
    E.g.: for the VREP opcode the M4 value determines which I2 values
    are allowed. This constraint cannot be expressed. However, the
    disassembly for such insns is not compared. Use --show-spec-exc
    to show those insns.

(4) In s390_decode_and_irgen the code peeks past the current insn:

    /* If next instruction is execute, stop here */
    if (dis_res->whatNext == Dis_Continue &&
        (bytes[insn_length] == 0x44 ||
         (bytes[insn_length] == 0xc6 && (bytes[insn_length + 1] & 0xf) == 0))) {

    Because of this we need to make our insn buffer 7 bytes instead
    of 6 and set insn_buf[6] = 0x0. This works because 0x0 != 0x44
    and 0x0 != 0xc6.
    Perhaps disable the peek-ahead when inside disasm-test by means
    of some global variable? Not pretty either, but explicit.

(5) For D20XB and D12XB operands add a test with B == X and B != 0
    Not exactly easy to do.
