\documentstyle{article}

\oddsidemargin 0in
\evensidemargin 0in
\textwidth 6.5in
\topmargin -0.5in
\textheight 9in

\title{Interfacing to TAU \\ {\large(draft)}}
\author{Lars Thomas Hansen}
\date{July 18, 1995}

\newcommand{\code}[1]{{\tt #1}}
\begin{document}
\maketitle

\section{Introduction}

The TAU tools rely on an interface to a compiler or to a compiler
toolkit to retrieve information about the program under analysis and to
instrument the program for debugging, profiling, and so on.

This document defines the interface between TAU and two compiler toolkit
frontends: \code{cgm} provides TAU with information about the program, and
\code{instr} is invoked by TAU to instrument the program. The interface
is defined in terms of data layouts and their semantics, command line
switches given to the programs, and the command languages accepted by
the programs.

In addition, we define the format of the output data which must be
produced by a program instrumented with \code{instr}.

\subsubsection*{Information retrieval}

TAU gets its data from the Sage++ toolkit by way of a program called
\code{cgm} (``call graph manager''). \code{Cgm} takes as arguments the
name of the program database and a specification of the desired output
format, and prints a digest of the call graph on standard output. TAU
reads and parses the output.

Section \ref{cgm} defines the output data format.

\subsubsection*{Instrumentation}

TAU instruments the user program by way of a program called \code{instr}
(``instrumenter''). \code{Instr} takes as arguments the type of
instrumentation to be performed (breakpointing or profiling) and, in the
case of profiling, a command file describing what functions to
instrument. It produces an output file with the desired data.

\section{Information Retrieval}
\label{cgm}

This section defines the data that any interface to a toolkit database
must supply, and the format this data must be supplied on.

The data ``must'' be supplied by a standalone program called
\code{cgm}. This program takes one output option and the name of the
program database (below called a depfile). It produces, on standard
output, the data on the required format.

The output options are \code{-dumpall}, \code{-dumpcg}, \code{-dumptxt},
\code{-dumpch}, \code{-dumpbdb}, and \code{-dumpdebug}.
These options are mutually exclusive and are described below.

TAU really should have a configuration file which specifies how it is
to obtain the various forms of data that it needs in terms of the programs
and options to use.

The Sage++ version of \code{cgm} can also be used stand-alone, but this does
not concern us in this paper.

\subsection{General requirements}

Some definitions:
\begin{itemize}
\item A {\em tag} is a nonnegative integer in the current implementation;
      it can be any string, but due to the uniqueness requirement (see
      below) some form of name mangling would have to be implemented if
      the program identifiers were to be used here.
\item A {\em string} is a sequence of nonblank characters.
\end{itemize}

Some general requirements and properties of the output:
\begin{itemize}
\item
All line offsets are 1-based; column offsets are 0-based.
\item
Class and function tags must be unique in the two important ways. First,
no tag may be used for two separate entities within its group; i.e., no
two functions can have the same tag, nor can two classes have the same
tag. There can exist a tag, however, which names both a function and a
class. Second, if two calls are made to \code{cgm} with the same
depfile, the tags produced by the two calls must be the same for the
same program elements.
\item
Lines may have any length.
\item
The spacing of fields is arbitrary: there must be at least one space to
separate fields, but the field can be preceded and succeeded by
arbitrary spaces.
\end{itemize}

\subsection{Option \code{-dumpall} -- dump call graph (all functions)}

The \code{-dumpall} option dumps a representation of all the functions 
in the depfile, whether used or not. Each function starts with an \code{ftag}
line and is followed by other information lines. Functions are separated
by a blank line. There may be only one line of each type for each function.

The first line of the output contains a single number: the number of functions
in the depfile.

Certain internal functions generated by the instrumentation code are
considered ``hidden'' and are not shown with this option; use
\code{-dumpdebug} (described below) to display also hidden functions.

\subsubsection{The \code{ftag} line format}

The \code{ftag} line is used to define a new function. The syntax of the
\code{ftag} line is:
\begin{verbatim}
      ftag: <tag> <name>
\end{verbatim}
\noindent
where \code{<tag>} is the tag of the function and \code{<name>} is the 
function's name (a string). 

\subsubsection{The \code{ffile} line format}

The \code{ffile} line specifies the location of the source code of the 
current function. The syntax of the \code{ffile} line is:
\begin{verbatim}
      ffile: <line> <filename>
\end{verbatim}
\noindent
where \code{<line>} is the line on which the function starts, and
\code{<filename>} is the name of the file in which the function is defined.
If either the line or the file name is not obtainable, use the format
\begin{verbatim}
      ffile: 0 -
\end{verbatim}

\subsubsection{The \code{fcalls} line format}

The \code{fcalls} line specifies calling information about the current 
function. The syntax of the \code{fcalls} line is:
\begin{verbatim}
      fcalls: <calls> - <tag> ...
\end{verbatim}
\noindent
where \code{<calls>} is the number of calls made to the function and
each \code{<tag>} is the tag of a function called from the current
function. The called function tags are listed in the sequence in which they
are called, and there will be repetitions if a function is called more
than once. See also the description for \code{fline}, below. The \code{-}
has to be present even if there are no tags.

\subsubsection{The \code{fclass} line format}

The \code{fclass} line is used to specify the class or collection of which
the current function is a member, if it is a member function. The syntax
is:
\begin{verbatim}
      fclass: <classname> <classtype> <classtag>
\end{verbatim}
\noindent
where \code{<classname>} is name of class if the function is a member
function, or \code{-} if not; \code{<classtype>} is either \code{-}
(ordinary class, or not applicable) or \code{COLL}; and \code{<classtag>}
is the tag of the class (see the section for the \code{-dumpch}, below)
or \code{-2} if the \code{<classname>} is \code{-}.

\subsubsection{The \code{ftype} line format}

The \code{ftype} line is used to specify some attributes of the function.
The syntax is:
\begin{verbatim}
      ftype: <exetype> <funtype> <usestat>
\end{verbatim}
\noindent
where \code{<exetype>} is either \code{seq} (sequential function) or
\code{par} (member function called in a parallel section, or Fortran M
\code{process}); \code{<funtype>} is one of \code{Mem} (member
function), \code{Ord} (ordinary function), \code{Prcs} (Fortran M
\code{process}), or \code{Main} (main function); and \code{<usestat>} is
\code{used} (a syntactically visible call to the function is reachable
from \code{main}) or \code{not} (unable to determine).

\subsection{Option \code{-dumpcg} -- dump call graph (called functions)}

The \code{-dumpcg} option is used to dump the functions in the call
graph, i.e., those functions in the depfile which are syntactically
referenced in a procedure call. The output format here is the same
as the one used with the \code{-dumpall} option, and in addition there
is one more line: the \code{fline} line.

The first line of the output contains a single number: the number of functions
in the depfile.

\subsubsection{The \code{fline} line format}

The \code{fline} line specifies the occurences of function calls within
the source text of the current function. The syntax of the \code{fline}
line is:
\begin{verbatim}
      fline: <line> ...
\end{verbatim}
\noindent
where each \code{<line>} is a line number offset from the start of the
file; the line is within the source text of the function and contains a
function call. There are as many function calls in this list as there
are called functions in the \code{fcalls} line (above), and the ordering
of the data in \code{fcalls} corresponds to the ordering in
\code{fline}.  If there are no line numbers to list because there were
no calls, the \code{fline} line is not printed.

\subsection{Option \code{-dumptxt} -- dump source locations}

The \code{-dumptxt} option is used to dump the functions in the call
graph along with information which allows TAU to find the source text
for the function in the source file. The output format is that of
\code{-dumpall}, except that the \code{ffile} line is more elaborate,
the \code{fline} line is replaced by a new line type \code{fpos}.

The first line of the output contains a single number: the number of functions
in the depfile.

The \code{-dumptxt} option also dumps the following class information as
formatted for the \code{-dumpcg} option: 
\code{ctag}, \code{cfile} (with additional source code line and column
positions as formatted for \code{ffile}), \code{cbase}, \code{cpos}, 
\code{cmpos} ({\bf THESE LINES NEED BETTER DESCRIPTIONS!}).



\subsubsection{New \code{ffile} syntax}

The new \code{ffile} syntax allows for more elaborate specification of the
position of the function in the source text. Compared with the old syntax, 
the line has appended four new fields which define the exact location of the
function's text. The syntax is:
\begin{verbatim}
      ffile: <line> <filename> @ <p1> <p2> <p3> <p4>
\end{verbatim}
\noindent
where each \code{<pi>} is of the form \code{<line>.<col>} and denotes a
line and a column within the source file. \code{<P1>} is the (location
of the) start of the function header; \code{<p2>} is the end of the
header; \code{<p3>} is the start of the body; and \code{<p4>} is the end
of the body. If for some reason the information could not be determined,
each \code{<pi>} is \code{-}. Each ending position specifies the character
following the end, not the character at the end.

\subsubsection{The \code{fpos} line format}

The \code{fpos} line is used to specify the exact location of procedure calls
within the source text of the function. The syntax is:
\begin{verbatim}
      fpos: <loc> ...
\end{verbatim}
\noindent
where each \code{<loc>} is a Tcl string of the format \verb+{ <pos1> <pos2> }+,
where each \code{<pos>} is either \code{-} or of the form \code{<line>.<col>};
here, \code{<line>} and \code{<col>} are nonnegative integers denoting line
and column information, respectively, in the file for a call; \code{<pos1>} 
is the start position of the identifier in the call, and \code{<pos2>} is the
first position following the identifier. If the function being called is
an operator, then both \code{<pos>} are currently \code{-} since the Sage++
toolkit does not keep location information for operator symbols; however,
they may be supplied and will be used.

\subsection{Option \code{-dumpbdb} -- dump data for the broswer database}

The \code{-dumpbdb} option dumps only the depfile information that is needed
by the browswer database.  This includes the \code{ftag} and
\code{ffile} lines as formatted for the \code{-dumpall} option, and the
\code{ctag} and \code{cfile} lines as formatted for the \code{-dumpch}
option.  Source code line and column positions are not written.

In addition, the \code{ftype} line is output if and only if the function
is the Main function of the program. The syntax is:
\begin{verbatim}
      ftype: Main
\end{verbatim}
\noindent


\subsection{Option \code{-dumpdebug} -- dump extra debug data}

When the \code{-dumpdebug} option is given, debugging information about
each function is dumped. The debugging information takes the same form
as the \code{-dumpall} information, except that the \code{fline} is no
longer present and that there are two types of extra information. First,
hidden functions are shown in the output. Second, a special line
\code{fdebug} containing debug data is dumped for each function.

\subsubsection{The \code{fdebug} line format}

The \code{fdebug} line provides some information not otherwise available.
It is used only for debugging, not by any of the tools, and need not be
supplied by new interfaces to compiler toolkits. It is documented here
for the sake of completeness. The syntax is:
\begin{verbatim}
      fdebug: moe=<bool> moc=<bool> seq=<bool> par=<bool> min=<int> max=<int>
\end{verbatim}
\noindent
Here, ``moe'' means ``method-of-element'', ``moc'' means
``method-of-class'', and ``seq'' and ``par'' mean what you think. A
\code{<bool>} is either \code{0} or \code{1}. ``Min'' is the minimum
nesting level in the call graph of the function, starting from the main
procedure, and ``max'' is the maximum nesting level. The main procedure
has nesting level 0.

\subsection{Option \code{-dumpch} -- dump the class hierarchy}

The \code{-dumpch} option is used to produce a representation of the class
hierarchy. 
The output is a sequence of class descriptions separated by blank
lines.  A class description consists of a \code{ctag} line to identify
the class, followed by a single \code{csub} line to list the subclasses,
and as many \code{cmem} lines as needed, one for each member. The class
hierarchy is dumped in preorder with a distinguished dummy root class as
an anchor.

\subsubsection{The \code{ctag} line format}

A \code{ctag} line serves to identify a class or collection. It has the
following form:

\begin{verbatim}
      ctag: <tag> <classname> <coll>
\end{verbatim}

\noindent
where \code{<tag>} is the tag of the class, or -1 if the line describes
the root of the hierarchy. \code{<Classname>} is either the empty Tcl
string \verb+{}+ for the root of the hierarchy or a string for real
classes or collections. \code{<Coll>} is \code{-} if the line describes
an ordinary class or \code{COLL} if the line describes a collection.

\subsubsection{The \code{csub} line format}

The \code{csub} line is used to specify the subclasses of a given class.
Its syntax is:

\begin{verbatim}
      csub: <subclasstag> ...
\end{verbatim}

\noindent 
where \code{<subclasstag>} is the tag of another class.

\subsubsection{The \code{cmem} line format}

A \code{cmem} line identifies a single member of a class or collection.
It has the following form:

\begin{verbatim}
      cmem: <member-ident> <member-tag> <member-bits>
\end{verbatim}

\noindent 
where \code{<member-ident>} is the name of the member (an identifier or
operator symbol), \code{<member-tag>} is a tag which identifies the
member, and \code{<member-bits>} is a Tcl string on the format
\verb+{abcde}+ where \code{a}--\code{e} are single characters with the
following values and meaning:

\begin{itemize}
 \item \code{a} and \code{b} describe the gross member type:
  \begin{itemize}
   \item \code{a} is either ``?'' (unknown), ``i'' (private), ``u'' (public),
         ``o'' (protected), or ``e'' (element).
   \item \code{b} is either ``f'' (function) or ``v'' (variable).
  \end{itemize}
 \item \code{c} is ``v'' if the member is virtual, else blank.
 \item \code{d} is ``i'' if the member is inline, else blank.
 \item \code{e} is ``c'' if the member is a constructor, ``d'' if the member
       is a destructor, ``o'' if the member is an operator, else blank.
\end{itemize}

\section{Instrumenting the Program}

\code{Instr} is a program which is used by the TAU tools to instrument
the program under inspection. By a combination of a simple command language 
and some command line switches the program can be instrumented for 
profiling and breakpointing with reasonable control over what parts
of the program are instrumented and which are not.

The command line switch \code{-breakpoint} causes \code{instr} to
instrument the program for breakpointing; the switch \code{-profile}
causes \code{instr} to instrument the program for profiling. In
addition, an instrumentation command file may be specified with the
\code{-commands} option.  The command file, if present, tells
\code{instr} which functions, files, and classes to include in or
exclude from the instrumentation.

\subsection{Command language}

The grammar for the command language is shown in figure \ref{cmdlfig}.

\newcommand{\nt}[1]{$<$#1$>$}                 % nonterminal
\begin{figure}[hbtp]
% Yes, this is not pretty.
\newcommand{\arrow}{\>$\Longrightarrow$\>\>}  % lhs-to-rhs arrow
\newcommand{\tbar}{\>\>$\mid$\>}              % bar under arrow
\newcommand{\vbar}{$\mid$\ }                  % bar in rhs
\renewcommand{\t}[1]{{\tt #1}}                % terminal
\newcommand{\q}{\\}                           % end-of-line
\renewcommand{\empty}{$\epsilon$\ }           % empty production
\renewcommand{\a}[1]{\>{\em ; #1}}            % annotation

\begin{tabbing}
sdfkasdfkasdlfkjsd \= cc \= c \= cccccccccccccccccccccccccccccccccccccccccccccc \= \kill
\nt{commands}    \arrow \nt{commandline} \nt{commands} \q
\nt{commandline} \arrow \nt{command} \nt{end-of-line} \q
\nt{command}     \arrow \nt{level-command} \q
                   \tbar \nt{inline-command} \q
                   \tbar \nt{limit-command} \q
                   \tbar \nt{exclude-command} \q
                   \tbar \nt{include-command} \q
                   \tbar \empty \q
\nt{level-cmd}  \arrow \t{include} \t{up} \t{to} \t{level} \nt{number} \q
\nt{inline-cmd} \arrow ( \t{exclude} \vbar \t{include} ) \t{inline} ( \nt{function-word} \vbar \empty ) \q
\nt{limit-cmd} \arrow \t{count} ( \t{only} \vbar \empty ) \nt{limit} \q
                \tbar \t{exclude} \nt{limit} \q
\nt{limit} \arrow \t{smaller} \t{than} \nt{number} \nt{stmt-word} \nt{logical} \nt{number} \nt{child-word} \q
 \tbar \t{smaller} \t{than} \nt{number} \nt{child-word} \nt{logical} \nt{number} \nt{stmt-word} \q
\nt{exclude-cmd} \arrow \t{exclude} \nt{parameter} \q
\nt{include-cmd} \arrow \t{include} \nt{parameter} \q
\nt{parameter} \arrow \t{file} \nt{name} \q
\tbar \t{class} \nt{name} \q
\tbar \nt{function-word} \nt{name} \q
\tbar \nt{qualified-name} \q
\nt{name} \arrow \nt{ident} \vbar \nt{regexp} \q
\nt{qualified-name} \arrow \nt{name} \t{:} \nt{name} \t{::} \nt{name} \a{file:class::function} \q
\tbar \nt{name} \t{::} \nt{name} \a{class::function} \q
\tbar \nt{name} \t{:} \nt{name} \t{::} \a{file:class::} \q
\tbar \nt{name} \t{:} \nt{name} \a{file:function} \q
\tbar \nt{name} \t{:} \a{file:} \q
\tbar \nt{name} \t{::} \a{class::} \q
\tbar \nt{name} \a{function} \q
\nt{logical} \arrow \t{and} \vbar \t{or} \q
\nt{stmt-word} \arrow \t{statement} \vbar \t{statements} \q
\nt{child-word} \arrow \t{child} \vbar \t{children} \q
\nt{function-word} \arrow \t{function} \vbar \t{functions} \q
\end{tabbing}
\caption{The \code{instr} command language grammar}
\label{cmdlfig}
\end{figure}

The \nt{level-cmd} command tells \code{instr} to instrument only those
functions in the call graph which have a call graph level of less than
or equal to what is specified, where the main function (in most cases,
\code{Processor\_main}) has a level of zero.

The \nt{limit-cmd} command has two variations. The \code{count}
variation tells \code{instr} to only insert instrumentation which counts
the number of calls to functions which are smaller than some limit,
rather than instrumentation which computes the time spent in those
functions (the default). The \code{exclude} variation excludes functions
smaller than the limit from instrumentation altogether.  In both cases,
the limit is measured in the number of statements in the function and
the number of function calls it contains.

The \nt{exclude-cmd} command is used to exclude whole classes of functions
based on either file, class, or function name. The \nt{include-cmd} command
is similarly used to include these classes of functions.

The singular or plural forms of the words \code{function}, \code{child},
and \code{statement} may be used interchangeably regardless of the
magnitude of the quantity specified. The word \code{only}, if present,
has no information content. The language is case-sensitive; only
lower-case keywords are recognized.

Any number of commands can be present in the commands file. However,
only the last instances of the \nt{level-cmd}, \nt{inline-cmd}, and
\nt{limit-cmd} commands will be used by \code{instr}. 

When functions, files, and classes are selected, rules which limit their
size or structure (e.g. \code{exclude inline}) are applied first, then
rules which selectively exclude and include based on name matching.  In
the case of contradictory instances of \nt{include-cmd} and
\nt{exclude-cmd} commands, rules of precedence apply. Entities specified
in an \code{include} statement are preferred over those in an
\code{exclude} statement, so one can focus on for example one single
file by excluding all files and then including the single desired file;
this works because the default is to include everything.

\subsection{Instrumentation}

This section presents, on a high level, the type of instrumentation
performed inserted by \code{instr}. Details are left out as they are
specific to the compiler toolkit being used.

\subsubsection{Profiling}

The profiling code currently works only for C++ programs; Fortran and C
are not supported at all.

The C++ class \code{Profiler} must be declared in the program,
do this by including the standard header file
\code{target/include/profiler.h}. 

In each function to be profiled, \code{instr} declares a local auto
variable of class \code{Profiler} with the name \code{pcxx\_P}. This
class has an overloaded constructor function; \code{instr} creates an
argument list for the constructor depending on which profiling options
are appropriate for the function (currently only whether the function is
to be profiled counting-only or not).

In addition, \code{Processor\_main} is instrumented specially with a
variable of class \code{Profiler} with a call to a constructor which
takes the number of functions to be profiled; the purpose of this
special case is to let that special constructor allocate space for the
profiler.

\subsubsection{Breakpointing}

[need something here]

\subsubsection{Tracing}

[something here??]

\section{Instrumentation Output Data Format}

\subsubsection{Profiler data output format (profile.ftab)}

Generated only when instrumenting for profiling.

The output from the profiler is encoded in ASCII with newline-terminated
lines; each datum is on a line by itself.

The first line contains the name of the depfile on which the
instrumentation was performed. The second line contains a count of the
number of functions profiled, plus 11.

Thereafter follows 11 lines on the format \verb+<tag>\t<name>+ where
\code{<tag>} is the tag of the function and \code{<name>} is its name;
if the tag is -1, then the name is blank and the line is to be
ignored. These lines describe system functions like the main procedure, the
barrier, polling, and data element lookup. 

Finally, there are lines on the format \verb+<tag>\t<name>-<count>+
where \code{<name>} is a function and \code{<count>} is the number of
times it was called. If the function was not called, the dash and the
count are omitted.

\subsubsection{Type description files}

Generated when instrumenting for breakpointing.

%\subsubsubsection{tdl format}

%\subsubsubsection{edf format}


\end{document}
