*****************************************************************************
**                      Tuning and Analysis Utilities                      **
**			http://www.cs.uoregon.edu/research/paracomp/tau    **
*****************************************************************************
**    Copyright 1997-2004                                                  **
**    Department of Computer and Information Science, University of Oregon **
**    Advanced Computing Laboratory, Los Alamos National Laboratory        **
**    Research Center Juelich, ZAM Germany                                 **
*****************************************************************************

*************************************************************************
*			USING TAU WITH MUSE	 	      		*	
* Author	: Suravee Suthikulpanit (suravee@cs.uoregon.edu)	*
* Date		: 3/2/2004
* Version	: 1.3							*
* Log Change	:							*
*		- Add multiple metrics support for monotonically 	*
*		increasing value.					*
*		- Found synchronization problem in MUSE and still	*
*		working on the solution.				*
*									*
*		Log 1.2							*
*		- Add user-defined event support for non-monotonically  *
*		increasing value.					*
*									*
*		Log 1.1							*
*		- (1) Restructure struct package to be ready for 	*
*		multiple double values, multiple handlers, 		*
* 		multiple filters.					*
*		- Add partial support for (1)				*
* 		- Move package_selector stuff into TauMusePackages.cpp	*
*		- Add API TauMuseCreate ()				*
* 		- Add package context_switch.				*
*		- Add package accumulator_sock_send.			*
*		- Add package accumulator_sock_recv.			*
*		- Add support for new socket_filter.			*
*		- Change encoding for process_scheduling		*
*		- Lengthen size of current_directory to fix a bug.	*
*************************************************************************

INTRODUCTION:

MAGNET: 
	MAGNET instruments Linux Kernel with a set of instrumentation points
	"MAGNET_ADD()" which will output records of specific kernel events 
	to userspace by MUSE.

MUSE: MAGNET User-Space Environment.
	MUSE contains the implementation of magnetd and utilities for monitoring
	MAGNET events.  Users can implement a set of "Handlers" and "Filters"
	to analyze event records information received from MAGNET.

	For more information, please refer to http://public.lanl.gov/radiant/

MUSE NOTE:
	- 

*************************************************************************
INSTALLATION:

% configure -muse		: Enable TAU_MUSE for single monotonically
				increasing value.
or 
% configure -muse_event		: Enable TAU_MUSE with multiple non-monotonically 
				increasing values.
or 
% configure -muse_multiple	: Enable TAU_MUSE with multiple monotonically 
				increasing values.

NOTE: -muse or -muse_multiple can be configured together with -muse_event i.e.

	% configure -muse -muse_event
   or 
	% configure -muse_multiple -muse_event
   
   The configurations will specify the uses of MAGNET/MUSE to extract low-level 
   information from the kernel. To use this configuration, Linux kernel has 
   to be patched with MAGNET and MUSE has to be install on the executing 
   machine.  Also, magnetd has to be running with the appropriate 
   handlers and filters installed.  

*************************************************************************

MODES OF OPERATION:

   -muse 

   This option is used when single monotonically increasing value is monitored. 

   		double TauMuseQuery() 

   is automatically called for every entry and exit point of every function 
   in a target application.  A monitored double value is returned while -1 
   is returned in case of error.

   User must specify environment variable "TAU_MUSE_PACKAGE" to a corresponding
   package.
	
   =============================================================================
   
   -muse_event 

   This option is used when multiple non-monotonically increasing values are 
   monitored.  Query is made through API
            
		int TauMuseEventQuery(double data[], int array_size) 

   It returns size of results in the array provided, and -1 in case of error.

   		int TauMuseGetMetricsNonMono(char *metricName[], int array_size) 

   is used to acquire metrics' name and order that are returned in the provided
   array. This function returns size of results in the array, and -1 in case 
   of error.
 
   User must specify environment variable "TAU_MUSE_EVENT_PACKAGEx" to a corresponding
   package where "x" is a consecutive number from 0 to 9.  An array of double
   and size of array must be specified.
	
   =============================================================================
   
   -muse_multiple

   This option is used when multiple non-monotonicall increasing values are 
   monitored.  Query is made through API
            
		int TauMuseMultipleQuery(double data[], int array_size) 

   It returns size of results in the array provided, and -1 in case of error.

   		int TauMuseGetMetricsMono(char *metricName[], int array_size) 

   is used to acquire metrics' name and order that are returned in the provided
   array. This function returns size of results in the array, and -1 in case 
   of error.
 
   User must specify environment variable "TAU_MUSE_PACKAGEx" to a corresponding
   package where "x" is a consecutive number from 0 to 9.  An array of double
   and size of array must be specified.
	
*************************************************************************

USAGE:

   TAU provides different packages which are predefined set of MUSE handlers and
   filters with specific arguments.  These packages let TAU users monitor 
   various performance metrics.
   
   =============================================================================
   LIST OF PACKAGES:
   =============================================================================
   MODE OF OPERATION: Monotonically increasing value(s).

   1) Package Name:	count
	This package keeps track of number of MAGNET instrumentation points executed
	in the LINUX kernel caused by the profiled program.  These counts are event 
	records according to the specified PID.  
	Handler			: TAU_count
	Filter			: process_filter
	Number of Metrics	: 1 
	Usage   		: Prior to execution, set enviroment variable 
				TAU_MUSE_PACKAGE=TAU_count or
				TAU_MUSE_PACKAGEx=TAU_count
	Command	:	
	create TAU_count
	addfilter <handlerID> process_filter pid=<pid>

   2) Package Name: 	busy_time
  	Total time in millisec of processors in BUSY state doing work 
	for the specified PID.
	Handler 		: process_scheduling
	Filter			: process_filter
	Number of Metrics	: 1 
	Usage 			: Prior to execution, set enviroment variable 
				TAU_MUSE_PACKAGE=busy_time or
				TAU_MUSE_PACKAGEx=busy_time
	Command	:
	create process_scheduling
	addfilter <handlerID> process_filter pid=<pid> event=258 //MAGNET_TASK_CTX_IN	
	addfilter <handlerID> process_filter pid=<pid> event=259 //MAGNET_TASK_CTX_OUT	

   3) Package Name: 	idle_time
  	Total time in millisec of processors in Idle state. 
	Handler 		: process_scheduling
	Filter	 		: process_filter
	Number of Metrics	: 1 
	Usage   		: Prior to execution, set enviroment variable 
				TAU_MUSE_PACKAGE=idle_time or
				TAU_MUSE_PACKAGEx=idle_time
	Command	:
	create process_scheduling
	addfilter <handlerID> process_filter pid=0 event=258 //MAGNET_TASK_CTX_IN	
	addfilter <handlerID> process_filter pid=0 event=259 //MAGNET_TASK_CTX_OUT	

   4) Package Name: 	total_time 
  	Total time in millisec of processors in execution for the 
	specified PID. 
	Handler 	 	: process_scheduling
	Filter			: process_filter
	Number of Metrics	: 1 
	Usage  			: Prior to execution, set enviroment variable 
				TAU_MUSE_PACKAGE=total_time or
				TAU_MUSE_PACKAGEx=total_time
	Command	:
	create process_scheduling
	addfilter <handlerID> process_filter pid=<pid> event=258 //MAGNET_TASK_CTX_IN	
	addfilter <handlerID> process_filter pid=<pid> event=259 //MAGNET_TASK_CTX_OUT	
	
   5) Package Name: 	context_switch
  	Total number of context switches for the specified PID. 
	Handler 		: process_scheduling
	Filter			: process_filter
	Number of Metrics	: 1 
	Usage   		: Prior to execution, set enviroment variable 
				TAU_MUSE_PACKAGE=context_switch or
				TAU_MUSE_PACKAGEx=context_switch
	Command	:
	create process_scheduling
	addfilter <handlerID> process_filter pid=<pid> event=258 //MAGNET_TASK_CTX_IN	
	addfilter <handlerID> process_filter pid=<pid> event=259 //MAGNET_TASK_CTX_OUT	

   6) Package Name: 	accumulator_sock_send
  	Total number of bytes sent out on sockets for a specified pid. 
	Handler 		: accumulator
	Filter			: socket_filter
	Number of Metrics	: 1 
	Usage   		: Prior to execution, set enviroment variable 
				TAU_MUSE_PACKAGE=accumulator_sock_send or
				TAU_MUSE_PACKAGEx=accumulator_sock_send
	Command	:
	create process_scheduling
	addfilter <handlerID> socket_filter=<pid> event=3 //MAGNET_SOCK_SEND

   7) Package Name: 	accumulator_sock_recv
  	Total number of bytes received through sockets for a specified pid. 
	Handler 		: accumulator
	Filter			: socket_filter
	Number of Metrics	: 1 
	Usage   		: Prior to execution, set enviroment variable 
				TAU_MUSE_PACKAGE=accumulator_sock_recv or
				TAU_MUSE_PACKAGEx=accumulator_sock_recv
	Command	:
	create process_scheduling
	addfilter <handlerID> socket_filter=<pid> event=4 //MAGNET_SOCK_RECV

   =============================================================================
   MODE OF OPERATION: Non-monotonically increasing value(s).

   1) Package Name:	bandwidth
	Average bandwidth (send and receive) in byte/sec is measured.	
	Handler			: bandwidth 
	Filter			: socket_filter
	Number of Metrics	: 2 
	Usage   		: Prior to execution, set enviroment variable 
				TAU_MUSE_EVENT_PACKAGEx=bandwidth
	Command	:	
	create bandwidth 1  //1 is the delta time for measuring average value.
	addfilter <handlerID> socket_filter pid=<pid>

   2) Package Name:	tcpbandwidth
	Average tcp bandwidth (send and receive) in byte/sec is measured.	
	Handler			: tcpbandwidth 
	Filter			: socket_filter
	Number of Metrics	: 2 
	Usage   		: Prior to execution, set enviroment variable 
				TAU_MUSE_EVENT_PACKAGEx=tcpbandwidth
	Command	:	
	create tcpbandwidth 1  //1 is the delta time for measuring average value.
	addfilter <handlerID> socket_filter pid=<pid>

*************************************************************************
 
NOTE: 
	Some configuration should be modified according to machine.
	- MAXNUMOFCPU 		: default to 2. 
	(This can run with Uni-processor and Dual-processor machine.)
	
	- AF_UNIX_MODE		: is defined by default.
	(This will configure muse to run in AF_INET or AF_UNIX mode. However, AF_INET is not
	recommend since it will significantly impact the performance of profiling.)
	
	- VAR_LOCK_DIRECTORY	: default to /var/lock 
	This will try to access magnetd file which is created in /var/lock.
 
*************************************************************************

BUGS:
	- Muse is having problem with DC and ST thread synchronization.
	- total_time is broken due to magnetd threads synchronization.
	- accumulator_sock_recv giving bogus value.

*************************************************************************

TODO:
