<?xml version='1.0'?>
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-
open.org/docbook/xml/4.5/docbookx.dtd" [
]>

<chapter id="userspace-probing">
	<title>User-space Probing</title>

<para>
SystemTap initially focused on kernel-space probing. Because
there are many instances where user-space probing can help diagnose
a problem, SystemTap 0.6 added support to allow probing user-space processes.
SystemTap can probe the entry into and return from a
function in user-space processes, probe predefined markers in
user-space code, and monitor user-process events.
</para>

<para>
SystemTap requires the uprobes module to perform user-space probing. If your
Linux kernel is version 3.5 or higher, it already includes uprobes.
To verify that the current kernel supports uprobes natively, run the following command:
</para>

<screen><command>grep CONFIG_UPROBES /boot/config-`uname -r`</command></screen>

<para>
If uprobes is integrated, the output of this command is as follows:
</para>

<screen>CONFIG_UPROBES=y</screen>

<para>
If you are running a kernel prior to version 3.5, SystemTap automatically
builds the uprobes module. However, you also need the utrace kernel
extensions required by the SystemTap user-space probing to track
various user-space events. More details about the utrace infrastructure are
available at <ulink url="http://sourceware.org/systemtap/wiki/utrace" />.
To determine whether the currently running Linux kernel provides the needed utrace support, type the following at a shell prompt:
</para>

<screen><command>grep CONFIG_UTRACE /boot/config-`uname -r`</command></screen>

<para>
If the Linux kernel supports user-space probing, the command produces the following output:
</para>

<screen>CONFIG_UTRACE=y</screen>

<!-- FIXME How does uprobes work for cross compilation and
 using a compile server? -->

<section id="uevents">
	<title>User-Space Events</title>

<indexterm>
<primary>Events</primary>
<secondary>user-space</secondary>
</indexterm>

<para>
All user-space event probes begin with <emphasis>process</emphasis>.
You can limit the process events to a specific running process by specifying the process ID.
You can also limit the process events to monitor a particular executable
by specifying the path to the executable (PATH).
SystemTap makes use of the <envar>PATH</envar> environment variable, which allows
you to use both the name used on the command-line to start the executable and
the absolute path to the executable.
</para>
<para>
Several of the user-space probe events limit their scope to a
particular executable name (PATH), because
SystemTap must use debug information to statically analyze where to
place the probes. But for many user-space probe events, the process ID
and executable name are optional. 
Any <command>process</command> event in the list below that include process ID or
the path to the executable must include those arguments.
The process ID and path to the executable are optional for the
<command>process</command> events that do not list them:
</para>

<variablelist>

<varlistentry>
<term>process("<replaceable>PATH</replaceable>").function("<replaceable>function</replaceable>")</term>
<listitem>
<para>
The entry to the user-space function <replaceable>function</replaceable> for the executable <replaceable>PATH</replaceable>.
This event is the user-space analogue of the
<command>kernel.function("<replaceable>function</replaceable>")</command> event.
It allows wildcards for the function <replaceable>function</replaceable>
and <literal>.return</literal> suffix.
</para>
</listitem>
</varlistentry>

<varlistentry>
<term>process("<replaceable>PATH</replaceable>").statement("<replaceable>statement</replaceable>")</term>
<listitem>
<para>
The earliest instruction in the code for <replaceable>statement</replaceable>. This is the user-space analogue of 
<command>kernel.statement("<replaceable>statement</replaceable>")</command>.
</para>
</listitem>
</varlistentry>

<varlistentry>
<term>process("<replaceable>PATH</replaceable>").mark("<replaceable>marker</replaceable>")</term>
<listitem>
<para>The static probe point <replaceable>marker</replaceable> defined in
<replaceable>PATH</replaceable>.
You can use wildcards for <replaceable>marker</replaceable> to specify
multiple marks with a single probe.
The static probe points may also have numbered arguments ($1, $2, and so on)
available to the probe.</para>
<para>A variety of user-space packages such as Java include these static
probe points.
Most packages that provide static probe points also
provide aliases for the raw user-space mark events.
Below is one such alias for the x86_64 Java hotspot JVM:
 </para>
<programlisting>probe hotspot.gc_begin =
  process("/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server/libjvm.so").mark("gc__begin")</programlisting>
</listitem>
</varlistentry>

<varlistentry>
<term>process.begin</term>
<listitem>
<para>A user-space process is created.
You can limit this to a particular process ID or a full path to the
executable.</para>
</listitem>
</varlistentry>

<varlistentry>
<term>process.thread.begin</term>
<listitem>
<para>A user-space thread is created.
You can limit this to a particular process ID or a full path to the
executable.</para>
</listitem>
</varlistentry>

<varlistentry>
<term>process.end</term>
<listitem>
<para>A user-space process dies.
You can limit this to a particular process ID or a full path to the
executable.</para>
</listitem>
</varlistentry>

<varlistentry>
<term>process.thread.end</term>
<listitem>
<para>A user-space thread is destroyed.
You can limit this to a particular process ID or a full path to the
executable.</para>
</listitem>
</varlistentry>

<varlistentry>
<term>process.syscall</term>
<listitem>
<para>A user-space process makes a system call.
The system call number is available in the <varname>$syscall</varname> context variable, and 
the fist six arguments are available in <varname>$arg1</varname> through <varname>$arg6</varname>.
The <literal>.return</literal> suffix places the probe at the return from the system call.
For <command>syscall.return</command>, the return value is available through the
<varname>$return</varname> context variable.</para>
<para>You can limit this to a particular process ID or a full path to the
executable.</para>
</listitem>
</varlistentry>

</variablelist>


</section>

<section id="utargetvariable">
	<title>Accessing User-Space Target Variables</title>

<indexterm>
<primary>target variables</primary>
<secondary>user-space</secondary>
</indexterm>

<para>
You can access user-space target variables in the same manner as described in
<xref linkend="targetvariables"/>.
In Linux, however, there are separate address spaces for the user and
kernel code.
When using the <literal>-&gt;</literal> operator, SystemTap accesses the appropriate address space.
</para>
<para>
For pointers to base types such as integers and strings,
there are a number of functions listed below to access user-space data.
The first argument for each functions is the pointer to the data item.
</para>

<variablelist>

<varlistentry>
<term>user_char(<replaceable>address</replaceable>)</term>
<listitem>
<para>
Obtains the character at <replaceable>address</replaceable> for the current
user process.
</para>
</listitem>
</varlistentry>

<varlistentry>
<term>user_short(<replaceable>address</replaceable>)</term>
<listitem>
<para>
Obtains the short integer at <replaceable>address</replaceable> for the current
user process.
</para>
</listitem>
</varlistentry>

<varlistentry>
<term>user_int(<replaceable>address</replaceable>)</term>
<listitem>
<para>
Obtains the integer at <replaceable>address</replaceable> for the current
user process.
</para>
</listitem>
</varlistentry>

<varlistentry>
<term>user_long(<replaceable>address</replaceable>)</term>
<listitem>
<para>
Obtains the long integer at <replaceable>address</replaceable> for the current
user process.
</para>
</listitem>
</varlistentry>

<varlistentry>
<term>user_string(<replaceable>address</replaceable>)</term>
<listitem>
<para>
Obtains the string at <replaceable>address</replaceable> for the current
user process.
</para>
</listitem>
</varlistentry>

<varlistentry>
<term>user_string_n(<replaceable>address</replaceable>, <replaceable>n</replaceable>)</term>
<listitem>
<para>
Obtains the string at <replaceable>address</replaceable> for the current
user process and limits the string to <replaceable>n</replaceable> bytes.
</para>
</listitem>
</varlistentry>

</variablelist>

</section>

<section id="ustack">
	<title>User-Space Stack Backtraces</title>

<indexterm>
<primary>Stack backtrace</primary>
<secondary>user-space</secondary>
</indexterm>

<!--
FIXME why is this useful?
FIXME describe how it works
FIXME cautions about debuginfo
FIXME when was -d -ldd options introduced?
FIXME description of how to use the stack backtracing
-->

<para>
      The probe point (<function>pp</function>) function indicates which
      particular event triggered the SystemTap event handler.
      A probe on the entry into a function would list the function name.
      However, in many cases the same probe point event may be triggered by
      many different modules in the program; this is particularly true for
      functions in shared libraries. A SystemTap backtrace of the user-space
      stack can provide additional context on how the probe point event is
      triggered.
</para>

<para>
      The user-space stack backtrace generation is complicated 
      by the compiler producing code optimized to eliminate stack frame
      pointers.
      However, the compiler also includes information in the debug information
      section to allow debugging tools to produce stack backtraces.
      SystemTap user-space stack backtrace mechanism makes use of that
      debug information to walk the stack to generate stack traces for 32-bit
      and 64-bit x86 processors; other processor architectures do not yet
      support the use of debug information to unwind the user-space stack.
			To ensure that the needed debug information is used to produce the
			user-space stack backtraces, use the
			<option>-d <replaceable>executable</replaceable></option> option for
			executables and <option>--ldd</option> for shared libraries.
</para>

<para>
      For example, you can use the user-space backtrack functions to see how
			the <function>xmalloc</function> function is
      being called by the <command>ls</command> command. With the
      debuginfo for the <command>ls</command> command installed, the following
      SystemTap command provides a backtrace each time the
      <function>xmalloc</function> function is called:
</para>

<screen><command>stap -d /bin/ls --ldd \</command>
<command>-e 'probe process("ls").function("xmalloc") {print_usyms(ubacktrace())}' \</command>
<command>-c "ls /"</command></screen>

<para>
When executed, this command produces output similar to the following:
</para>

<screen>
bin	dev   lib	  media  net	     proc	sbin	 sys  var
boot	etc   lib64	  misc	 op_session  profilerc	selinux  tmp
cgroup	home  lost+found  mnt	 opt	     root	srv	 usr
 0x4116c0 : xmalloc+0x0/0x20 [/bin/ls]
 0x4116fc : xmemdup+0x1c/0x40 [/bin/ls]
 0x40e68b : clone_quoting_options+0x3b/0x50 [/bin/ls]
 0x4087e4 : main+0x3b4/0x1900 [/bin/ls]
 0x3fa441ec5d : __libc_start_main+0xfd/0x1d0 [/lib64/libc-2.12.so]
 0x402799 : _start+0x29/0x2c [/bin/ls]
 0x4116c0 : xmalloc+0x0/0x20 [/bin/ls]
 0x4116fc : xmemdup+0x1c/0x40 [/bin/ls]
 0x40e68b : clone_quoting_options+0x3b/0x50 [/bin/ls]
 0x40884a : main+0x41a/0x1900 [/bin/ls]
 0x3fa441ec5d : __libc_start_main+0xfd/0x1d0 [/lib64/libc-2.12.so]
 ...
</screen>

<para>
      For more details on the functions available for user-space stack
      backtraces, refer to <filename>ucontext-symbols.stp</filename> and
      <filename>ucontext-unwind.stp</filename> tapsets. You can also find
			the description of the functions in the aforementioned tapsets in the
      <citetitle>SystemTap Tapset Reference Manual</citetitle>.
</para>


</section>

</chapter>
