18—
Windows NT Services

And feed Event Log records to it:

>>> win32evtlogutil.FeedEventLogRecords(CheckRecord)
Application [DCCMAN.EXE]. The application was started while the service manager
was locked and NtLmSsp wasn't running.
If the application will try to remote via WinSock Proxy it can cause a deadlock
with the service manager.
[and lots more boring stuff!]

Windows NT Performance Monitor Data

The Windows NT Performance Monitor is a tool and API that allows applications to provide performance data in a consistent way. Administrators can use the Performance Monitor tool to view this data graphically, or programs can themselves use this data for more advanced purposes, such as taking daily samples of performance data and logging to a database.

Using the Performance Monitor has a number of benefits for both the administrator and programmer. The administrator can use a single, consistent interface for monitoring performance, regardless of the application in question; indeed, the administrator can simultaneously view performance information from two unrelated applications to assist in diagnosing performance bottlenecks. For you, the programmer, the biggest advantage is that you can provide detailed performance information for your application, but don't need to deliver any tools for viewing or analyzing this data; Windows NT and third-party tools provide all the functionality anyone could need!

In the next section we discuss the concepts behind the Performance Monitor, and how you can read Performance Monitor data using Python. Later in this chapter we will present how Python can provide Performance Monitor data for our sample Windows NT Service.

Anatomy of the Performance Monitor

To use the Performance Monitor, an application must register one or more Performance Monitor objects. An object is a logical grouping of Performance Monitor data; it is quite common for an application to have a single object, grouping all performance-related data from the application in this single object.

For each object, the application must register one or more counters. A counter provides a single piece of performance data. Attributes about a counter include the units and the default scale for the final value. The units indicate how the counter values are to be transformed (for example, turned into a rate per second), while the scale defines a multiplication factor applied before the value is displayed or graphed.

For example, let's assume the application needs to keep a counter for connections per second made to it. The application is fast, so expect a hit rate in the thousands of connections per second. You would define your counter as having units of "per second" and a scale of "divide by 100." All the application needs to do is increment the counter once for each connection established, and the Performance Monitor itself will handle the transformation of the raw counter data to a value in the range 1 to 100 that represents the connections per second. The Performance

Monitor has kept track of how many counter increments were made in the last second and applied the appropriate conversions.

As you can see, from the application's point of view it's quite simple and unobtrusive; once the counters are defined, simply increment a counter whenever you do something interesting. The Performance Monitor manages the translation to useful units. For the program that wishes to view Performance Monitor information, it's also simple: the Performance Monitor itself has already translated the raw values into user-friendly values ready to display. Unfortunately, as we shall see later, the installation and definition of these counters isn't particularly easy, but once that is mastered, the actual usage of the counters in your application is simple.

In addition to objects and counters, the final Performance Monitor concept we mention is that of an instance. A counter can optionally have any number of instances associated with it. An instance is used where a single counter applies to multiple things. The most basic example is the Performance Monitor data for the CPU in your computer. Windows NT defines the object and a few counters for data related to a CPU. However, as there may be multiple CPUs installed in a PC, each counter can have an instance, one for each CPU. Thus, when you need to collect performance data for a specific CPU, you need to know the instance you are interested in, or you can ask to collect data for all instances. Most counters provided by applications don't have instances associated with them. In our previous example where we kept total client connections, no instance would be associated with the counter. In all the Performance Monitor examples that follow, we don't use counters with instances, but the concept is mentioned for completeness.

To see these concepts, the simplest thing is to start the Windows NT Performance Monitor (from the Administrative Tools group under the Windows NT Start menu), and select Edit ® Add to chart. In the dialog that's presented, you can clearly see the list of objects, counters, and instances. As you select different objects, the available counters will change. Depending on the counter you select, instances may or may not be available. Python can also display this list, so we will move on to Python and the Performance Monitor.

Reading the Performance Monitor from Python

Python exposes the ability to read Performance Monitor information from the win32pdh and win32pdhutil modules. These modules use a Microsoft API to access the Performance Monitor known as the Performance Data Helper, or PDH, hence the name of the Python modules.

Browsing for Counters

To get a feel for these Python modules, let's start with a demonstration of displaying a dialog for the user to browse and select the counters on this or any machine.

The win32pdhutil module provides a function browse() that displays such a dialog. As the user selects the Add button in the dialog, a function you provide is called with the selected counter. This callback function is supplied as a parameter to win32pdhutil.browse(). The first step is to provide the callback function, which takes a single parameter—the name of the counter. Our example simply prints this name.

Thus, the callback function can be defined as follows:

>>> def CounterCallback( counter ):
?nbsp;    print "Counter is", counter
?BR> >>>

You can display the dialog by importing the win32pdhutil module and calling the browse() function passing the callback:

>>> import win32pdhutil
>>> win32pdhutil.browse(CounterCallback)

A dialog is presented that allows you to select all the counters available on the system and even other systems! Select the Add button and your function is called, printing the selected counter. Figure 18-1 shows the code running under PythonWin, just after selecting the Add button.

The counter definition is a simple string, with the machine, object, counter, and instance portions-all embedded in the string. The win32pdh module contains functions for parsing a string of this format into the constituent parts and for building such a string from the parts; see the win32pdhutil module source code for more details.

Once you've determined the counters you want, obtaining the information is quite simple: use the win32pdhutil.GetPerformanceAttributes() function. In the simple case of a counter on the local machine without instances, you need pass only the object name and the counter name. For example, an object called Memory provides a number of counters, including one called Available Bytes. To obtain the value, use the code:

>>> win32pdhutil.GetPerformanceAttributes("Memory", "Available Bytes")
18358272
>>>

This demonstrates an interesting use of the Performance Monitor. There is some information about the system that's not easily available from the Windows API, but is available from the Performance Monitor. Statistics about processes (including its process ID) or about the memory in the system can often be obtained in far more detail by using the Performance Monitor than by using the API.

Writing Services in Python

We have already discussed how Python can control services, now let's see how to write services in Python. Services written in Python are first-class services and provide all the functionality of services written in C.

The Anatomy of a Service

Before we launch into how to write a service in Python, we must discuss some important service concepts. This will help you understand some of the design and implementation decisions made for Python services.

Windows NT starts a service by executing a process. Once this process starts, it's expected to report to the SCM that it's indeed a service and that it's starting. It also must pass to the SCM a control handler; that is, a function that responds to control messages sent by the SCM. The service process then executes. When the service is to be stopped, the SCM notifies the control handler of the stop request. The service itself is responsible for handling this request and terminating itself.

This leads to most services, regardless of the language they are written in, being structured as follows: A main thread starts. This thread initializes itself by reporting to the SCM and passing it the control handler. Once initialization is complete, the thread starts a worker loop waiting either for work to do or a notification to stop, pause, etc. The SCM calls the control handler on another thread, so once the function receives the control notification, it reports back to the SCM it's stopping, then triggers some synchronization object that the main thread will notice next time around the loop. As the main thread terminates, it continues to report its status to the SCM.

The Anatomy of a Python Service

The same executable hosts most Python services, PythonService.exe. This special executable is aware of the special requirements for services that make Python.exe unsuitable for the task.

When PythonService.exe is asked to start a service, it looks in the registry for the Python class that implements the service. It then creates an instance of this class, and delegates all service functionality to this instance. For example, when the service is to start, a method named SvcStart is called on the object. When the SCM makes a control request, a method named ServiceControlHandler is called. These Python methods are expected to correctly report their status to the SCM, by calling helper functions provided by PythonService.exe.

To make life as simple as possible for the Python programmer, a base class ServiceFramework is provided in the module win32serviceutil. The easiest way to write a service in Python is to subclass this class, then concentrate on the service functionality rather than on the interactions with the SCM.

The win32serviceutil.ServiceFramework Class

As described previously, most Python services will be a subclass of the win32serviceutil.ServiceFramework class. This has a number of default methods and helper methods to make writing services in Python easier:

__init__
The constructor for the class. This registers the method ServiceCtrlHandler as the handler for notification messages from the SCM.

ServiceCtrlHandler
Provides a default implementation of the service control handler. This method interrogates the class for certain named methods to determine which controls the service responds to. For example, if the service contains a SvcPause method, it's assumed the service can be paused.

SvcRun
A default implementation of the entry point for the service. This method notifies the SCM that the service has started, calls a method SvcDoRun, then when complete notifies the SCM the service is stopping (PythonService.exe automatically notifies the SCM that the service has stopped). Thus, you need only provide a SvcDoRun method in the subclass that handless the functionality of your service.

The Smallest Possible Python Service

Before we move on to a more substantial service written in Python, let's write the smallest possible service in Python. This service will do no actual work other than interact with the SCM.

The key points from this example code are:

?The logic for stopping the service must be provided by your application. This sample service uses a Win32 event object and when the command to stop the service is received, it sets this event. The service itself does nothing other than wait for this event to be set.

?The name of the service and the display name of the service must be provided by the subclass.

?This code contains startup code that handles the command line when run as a script. This provides facilities for installing, debugging, and starting the service, as described in the next section.

# SmallestService.py
#
# A sample demonstrating the smallest possible service written in Python.

import win32serviceutil
import win32service
import win32event

class SmallestPythonService(win32serviceutil.ServiceFramework):
    _svc_name_ = "SmallestPythonService"
    _svc_display_name_ = "The smallest possible Python Service"
    def __init__(self, args):
        win32serviceutil.ServiceFramework.__init__(self, args)
        # Create an event which we will use to wait on.
        # The "service stop" request will set this event.
        self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)

    def SvcStop(self):
        # Before we do anything, tell the SCM we are starting the stop process.
        self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
        # And set my event.
        win32event.SetEvent(self.hWaitStop)

    def SvcDoRun(self):
        # We do nothing other than wait to be stopped!
        win32event.WaitForSingleObject(self.hWaitStop, win32event.INFINITE)
if __name__=='__main__':
    win32serviceutil.HandleCommandLine(SmallestPythonService)

Installing, Debugging, and Running a Python Service

Now that we have a service written in Python, what to do with it? This section discusses installing the service, debugging the service, and starting and stopping the service.

Installing the Service

As discussed previously, an application named PythonService.exe hosts all Python services. This executable must be registered before you can install the service. The registration process need only be run once per machine, regardless of how many services the program hosts. The installation package for the Python for Windows extensions automatically registers this, but the information is included here for completeness. To register PythonService.exe, perform the following steps:

1. Start a command prompt.

2. Change to the directory containing PythonService.exe, typically \Program Files\Python\win32.

3. Execute the command:

C:\Program Files\Python\win32>PythonService.exe /register
Registering the Python Service Manager?BR> C:\Program Files\Python\win32>

Now to install the service, perform the following steps:

1. Start a command prompt.

2. Change to the directory containing the Python source code that implements the service.

3. Execute the command:

C:\Scripts> SmallestService.py install
Installing service SmallestPythonService to Python class
   C:\Scripts\SmallestService.SmallestPythonService
Service installed
C:\Scripts>

The service is now installed and ready to run. To confirm the service has been correctly installed, use the control panel to start the Services applet and scroll down until you find the ''The smallest possible Python Service.''

Starting and Stopping the Service

There are a number of ways to start or stop a service. Our Python script can start and stop itself. To do this, use the following command:

C:\Scripts> python.exe SmallestService.py start
Starting service SmallestPythonService
C:\Scripts>

The service is now running. To confirm this, let's try executing the same command again:

C:\Scripts> python.exe SmallestService.py start
Starting service SmallestPythonService
Error starting service: An instance of the service is already running.
C:\Scripts>

As you can see, only one instance of a service can be running at any time. To stop the service, use the following command:

C:\Scripts> python.exe SmallestService.py stop
Stopping service SmallestPythonService
C:\Scripts>

There are two other common techniques for starting and stopping services:

?Using the Services applet in the control panel, which provides a GUI for starting, stopping, or pausing services.

?Using the net.exe program supplied with Windows NT. From a Windows NT command prompt, this command starts the service:

C:\Anywhere>  net start SmallestPythonService

This command stops the service:

C:\Anywhere> net stop SmallestPythonService

Sample Service Written in Python

Before we move to some of the advanced topics, we will develop the basis for a real Python service that actually does something useful!

The first version of our service starts by accepting connections over a named pipe and comes complete with a client that connects to the service. You then enhance the service by writing to the Event Log and by providing Performance Monitor data.

The first cut looks very much like the SmallestPythonService, except it has more meat in the SvcDoRun() method. The main thread creates a named pipe and waits for either a client to connect or a service control request.

More information on named pipes can be found in Chapter 17, Processes and Files. This example also shows a number of concepts important when using named pipes. It shows how to use overlapped I/O, and how to create a special security object useful for named-pipe services:

# PipeServicel.py
#
# A sample demonstrating a service which uses a
# named-pipe to accept client connections.

import win32serviceutil
import win32service
import win32event
import win32pipe
import win32file
import pywintypes
import winerror

class PipeService(win32serviceutil.ServiceFramework):
    _svc_name_ = "PythonPipeService"
    _svc_display_name_ = "A sample Python service using named pipes"
    def __init__(self, args):
       win32serviceutil.ServiceFramework.__init__(self, args)
       # Create an event which we will use to wait on.
       # The "service stop" request will set this event.
       self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
       # We need to use overlapped 10 for this, so we don't block when
       # waiting for a client to connect. This is the only effective way
       # to handle either a client connection, or a service stop request.
       self.overlapped = pywintypes.OVERLAPPED()
       # And create an event to be used in the OVERLAPPED object.
       self.overlapped.hEvent = win32event.CreateEvent(None,0,0,None)

    def SvcStop(self):
       # Before we do anything, tell the SCM we are starting the stop process.
       self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
       # And set my event.
       win32event.SetEvent(self.hWaitStop)

    def SvcDoRun(self):
       # We create our named pipe.
       pipeName = "\\\\.\\pipe\\PyPipeService"
       openMode = win32pipe.PIPE_ACCESS_DUPLEX | win32file.FILE_FLAG_OVERLAPPED
       pipeMode = win32pipe.PIPE_TYPE_MESSAGE

       # When running as a service, we must use special security for the pipe
       sa = pywintypes.SECURITY_ATTRIBUTES()
       # Say we do have a DACL, and it is empty
       # (ie, allow full access!)
       sa.SetSecurityDescriptorDacl ( 1, None, 0 )

       pipeHandle = win32pipe.CreateNamedPipe(pipeName,
           openMode,
           pipeMode,
           win32pipe. PIPE_UNLIMITED_INSTANCES,
           0, 0, 6000, # default buffers, and 6 second timeout.
           sa)
       # Loop accepting and processing connections
       while 1:
           try:
               hr = win32pipe.ConnectNamedPipe(pipeHandle, self.overlapped)
           except error, details:
               print "Error connecting pipe!", details
               pipeHandle.Close()
               break
           if hr==winerror.ERROR_PIPE_CONNECTED:
               # Client is fast, and already connected - signal event
               win32event.SetEvent(self.overlapped.hEvent)
           # Wait for either a connection, or a service stop request.
           timeout = win32event.INFINITE
           waitHandles = self.hWaitStop, self.overlapped.hEvent
           rc = win32event.WaitForMultipleObjects(waitHandles, 0, timeout)
           if rc==win32event.WAIT_OBJECT_0:
               # Stop event
               break
           else:
               # Pipe event - read the data, and write it back.
               # (We only handle a max of 255 characters for this sample)
               try:
                   hr, data = win32file.ReadFile(pipeHandle, 256)
                   win32file.WriteFile(pipeHandle, "You sent me:" + data)
                   # And disconnect from the client.
                   win32pipe.DisconnectNamedPipe(pipeHandle)
               except win32file.error:
                    # Client disconnected without sending data
                    # or before reading the response.
                    # Thats OK - just get the next connection
                    continue
if __name__=='__main__':
    win32serviceutil.HandleCommandLine(PipeService)

This technique for working with named pipes doesn't scale well; our version accepts only a single client connection at a time. Another alternative is to use a new thread to process each connection as it comes in, as demonstrated in pipeTestService.py, which comes with the Window's extensions. Even this solution doesn't scale when the number of connections starts to become large, and other techniques, such as thread pooling or NT Completion Ports should be used.

Now let's write a client program to use this service. The client program is quite simple, because the Win32 API function CallNamePipe() encapsulates all of the code most clients ever need. CallNamedPipe() is available in the win32pipe module.

The client sends all the command-line parameters to the server, then prints the server's response:

# PipeServiceClient.py
#
# A client for testing the PipeService.
#
# Usage:
#
#   PipeServiceClient.py message

import win32pipe
import sys
import string

if __name__=='__main__':
    message = string.join(sys.argv[1:])
    pipeName = "\\\\.\\pipe\\PyPipeService"
    data = win32pipe.CallNamedPipe(pipeName, message, 512, 0)
    print "The service sent back:"
    print data

Now let's test the service. The first step is to register the service:

C:\Scripts> PipeService1 install
Installing service PythonPipeService to Python class
   C:\Scripts\PipeService1.PipeService
Service installed
C:\Scripts>

Now start the service:

C:\Scripts> PipeService1.py start
Starting service PythonPipeService
C:\Scripts>

And use the client to send the service some data:

C:\Scripts> PipeServiceClient.py Hi there, how are you
The service sent back:
You sent me:Hi there, how are you
C:>Scripts>

Our service seems to be working as expected!

You're now finished; you can stop the service:

C:\Scripts> PipeService1.py stop
Stopping service PythonPipeService
C:>Scripts>

And remove it from the Service database:

C:\Scripts> PipeService1.py remove
Removing service PythonPipeService
Service removed
C:\Scripts>

Writing to the Event Log

Almost all services use the Windows NT Event Log. It's quite common for services to load startup and shutdown events, and error conditions should always go to the Event Log.

We look first at the facilities provided by the servicemanager module for working with the Event Log. Then we will have another look at a couple of modules described previously in this chapter, win32evtlog and win32evtlogutil. We discussed how to use these modules to read the Event Log, so now let's look at how they can write to the Log.

The servicemanager Module

As mentioned previously in this chapter, PythonService.exe hosts all Python programs running as services. Once this host .exe is running, it makes a module servicemanager available to the Python program. Because your Python scripts are used both by Python.exe (when installing, starting, or debugging the service) and by PythonService.exe (when running as a service), you can't import this module from the top level of your Python program; the import servicemanager statement will fail for Python.exe. In practice, this means you should only import this module locally to a function.

The primary purpose of servicemanager is to provide facilities for interacting with the SCM at runtime. Most of this interaction is done by the base class win32serviceutil.ServiceFramework(), so in general this is covered for you.

There are, however, some useful utility functions you may wish to use. Among these is a set of functions for writing to the Event Log. The most general purpose is the function LogMsg(), which takes the following parameters:

ErrorType
One of EVENTLOG_INFORMATION_TYPE, EVENTLOG_ERROR_TYPE, or EVENTLOG_WARNING_TYPE. These constants can be found in servicemanager.

ErrorID
The message ID. This uniquely identifies the message text.

Inserts = None
A list of string inserts for the message or None. The inserts are merged with the message text to provide the final result for the user.

There are three other Event Log-related functions in this module. LogInfoMsg(), LogErrorMsg(), and LogWarningMessage(). All take a single string parameter and write a generic message to the Event Log. These functions also provide an additional debugging facility. When debugging the service, these Event Log helpers write their output to the console window. This allows you to see the messages your application would log without needing to use the Event Log Viewer. When the service is running normally, these message go to the log. We'll see this in action for our example.

The Event Log facilities of servicemanager use the name of the executable as the application name. This means that by default, the application name will be PythonService. The only way this can be changed is to take a copy of PythonService.exe and rename it to something of your liking (it's only around 20 KB!). Alternatively, you may wish to use the Event Log natively, as described later in this chapter.

Our host .exe also contains a number of messages related to starting and stopping the services. The servicemanager exposes the message ID of these through constants that include PYS_SERVICE_STARTING, PYS_SERVICE_STARTED, PYS_SERVICE_STOPPING, and PYS_SERVICE_STOPPED. These messages all have two inserts: the name of the service and an arbitrary message to be appended.

Modifying the Service to Add Messages

Finally we are ready to change our service to use the Event Log. We will change the service to write messages as it starts and stops, and also after each user has connected.

We add the code for the service starting message at the start of the service's SvcDoRun() method. A good place to add the service stopping message is just as this function returns.

The top of this function now looks like:

def SvcDoRun(self):
       # Log a "started" message to the event log.
       import servicemanager
       servicemanager.LogMsg(
               servicemanager.EVENTLOG_INFORMATION_TYPE,
               servicemanager.PYS_SERVICE_STARTED,
               (self._svc_name_, "))

And the last lines are:

# Now log a "service stopped" message
servicemanager.LogMsg(
        servicemanager.EVENTLOG_INFORMATION_TYPE,
        servicemanager.PYS_SERVICE_STOPPED,
        (self._svc_name_, "))

Note that you import the servicemanager object at the start of the function. This is because, as described previously, it's provided by the host .exe so it can't be imported at the top level of the program. To write a message as the user disconnects insert this code:

# Log a message to the event log indicating what we did.
message = "Processed %d bytes for client connection" % len(data)
servicemanager.LogInfoMsg(message)

Let's see our code in action. First, let's run this service in debug mode. From a command prompt, type:

C:\Scripts> PipeService2.py debug
Debugging service PythonPipeService
Info 0x40001002 - The PythonPipeService service has started.

Now make a client connection to the pipe. It's necessary to open a new command prompt, as the service is being debugged in the other one. From this new command prompt, type:

C:\Scripts> PipeServiceClient.py Hi there
The service sent back:
You sent me:Hi there

If you look at your service's window, you see the status message:

Info 0x400000FF - Processed 8 bytes for client connection

Finally, stop the service by pressing Ctrl-Break, and the service responds:

Stopping debug service.
Info 0x40001004 - The PythonPipeService service has stopped.

As you can see, the service works as expected, and now you can run it as a real service. This forces the messages to the Event Log.

Start the service using any of the techniques described previously, then make a new client connection. If you check the Event Log Viewer, you see the messages at the top of the application log. Figure 18-2 shows the Event Log Viewer with the list of all events; Figure 18-3 shows the detail for the client connection message.

Custom Event Log Writing

There will be cases where you need better facilities than those provided by the servicemanager module. The primary reason is that you will require your own set of messages, tailored for your application. In this scenario, you need the message compiler from Microsoft Visual C++ to create your own custom message texts and IDs.

Registering with the Event Log

The Event Log doesn't store the full text of each message in the log; instead, when the text is needed, it uses the ID of the message to look it up from a DLL or EXE file nominated by the application. This means you need to tell the Event log where your messages are located.

Applications register with the Event Log by writing an entry to the Event Log's registry. If you open the registry editor, you can view all applications providing Event Log data by looking under the key KLM\SYSTEM\CurrentControlSet\Services\EventLog.

Fortunately, the win32evtlogutil module has a utility function for registering your application. As the registry is persistent, it's necessary to register your application only when it's installed; however, some applications choose to do this each time they start.

Any EXE or DLL can hold messages for the Event Log Viewer. The text for messages are typically created with a text editor, and the "message compiler" (supplied with Microsoft Visual C++) compiles them. When the application is linked, the compiled messages are inserted into the final executable. However, this doesn't help you use the Event Log from Python, so a number of the standard Win32 extension files have built-in generic messages for this purpose.

The files PythonService.exe, win32evtlog.pyd, and win32service.pyd each have a set of generic messages. Each file has nine messages for each of the classes Error, Informational, Warning, and Success.

To register with the Event Viewer, use the function win32evtlogutil.AddSourceToRegistry().

win32evtlogutil.AddSourceToRegistry(ApplicationName, MessageDll, EventLogType= 揂pplication?/I>)

ApplicationName
The name of the application, as shown in the Event Log Viewer. This is an varbitrary name that need not match the name of the service.

MessageDll
The name of the Dll or EXE file that contains the messages.

EventLogType = "Application"
The Event Log this application writes to. Valid values are Application (the default), System, or Security.

Writing to the Log

Now that we are registered with the Event Log, it's time to start logging some messages. As mentioned before, you call the win32evtlog and win32evtlogutil modules to help out.

To manually insert a message in the Event Log, use the function win32evtlogutil.ReportEvent().

win32evtlogutil.ReportEvent(ApplicationName, EventID, EventCategory, EventType,
Inserts=[], Data=None, SID=None)

ApplicationName
The name of the application, as registered using the AddSourceToRegistry() function.

EventID
The numeric ID of the event.

EventCategory = 0
The event category, as defined by the message file.

EventType = win32evtlog.EVENTLOG_ERROR_TYPE
The numeric event type. This indicates the message is an error, a warning, informational, etc.

Inserts = []
A list of string inserts for the message. These inserts create the full text of the message and are inserted into the message text at the appropriate place holders.

Data = None
The data for the message. This is raw binary data as a Python string and is displayed in hexadecimal by the Event Log Viewer.

SID = None
A security identifier for the user this message relates to, if any.

Let's look at the long way we could add the startup and shutdown messages to our service. This should give you a reasonable idea of how you could implement your own custom messages in the Log.

If you use the table in Appendix B, you can see that PythonService.exe has messages specifically for this purpose: Informational Event IDs 1002 and 1004, respectively. The servicemanager module exposes these constants as PYS_SERVICE_STARTED and PYS_SERVICE_STOPPED. The table also indicates that this message requires two inserts: the first is the name of the service, and the second is arbitrary text you can append to the message. To log one of these messages, use the following code at the top of your SvcDoRun() method:

def SvcDoRun(self):
    # Log a "started" message to the event log.
    import servicemanager
    win32evtlogutil.ReportEvent(
            self._svc_name_, # Application name
            servicemanager.PYS_SERVICE_STARTED, # Event ID
            0, # Event category
            win32evtlog.EVENTLOG_INFORMATION_TYPE,
            (self._svc_name_, "")) # The inserts

This assumes you have already called win32evtlogutil.AddSourceToRegistry(), using the service name (as defined in _svc_name_) as the Event Log application name.

Providing Performance Monitor Information

The next step in our service will be to generate performance information that can be used by the Windows NT Performance Monitor.

The Performance Monitor itself is a fairly complex beast with many options. We will attempt to cover only the basics, enough to install a couple of counters into our sample service. If you require more advanced techniques, you should see the Win32 SDK documentation on the Performance Monitor.

The Performance Monitor has been designed to have minimal impact on the applications providing data. Applications provide raw data (by way of counters), and the responsibility of formatting and translating that data falls to the application processing the data (e.g., the Performance Monitor application provided with Windows NT). When a counter is added to the Performance Monitor, it indicates the type of data it represents. For example, one counter may indicate a raw count, such as total connections made, or it may be a value that should be interpreted over time, such as bytes sent per second. In both cases, the application increments a counter, and the Performance Monitor itself determines how far the counter was incremented in the last second and calculates the actual value to be displayed.

Counters are grouped together to form a Performance Monitor object. These objects exist purely to group related counters together. When you use the NT Performance Monitor Application, you must first select the object you wish to obtain data on, then the counters for that object. Windows NT comes with a number of objects and counters that obtain the performance of the operating system itself. For example, there is a standard processor object, and this object has counters such as percent of processor time, percent of interrupt time, and so forth.

The Performance Monitor expects to obtain the counter and object information from a special DLL. This DLL has public entry-points specific to Performance Monitor, and this DLL is loaded whenever the Performance Monitor needs information on the specific object. In general, this DLL is not an integral part of the application itself, but is instead a custom DLL written expressly for the Performance Monitor. Memory mapped files are typically used to share the counter data between the application itself and the Performance Monitor DLL.

When the application is installed, it must provide the Performance Monitor with definitions of the objects and counters it provides. These definitions include a name for the counter, and a description for the counter. The application must also provide the name of the DLL that provides the data.

Performance Monitor Data from Python

The Python Win32 Extensions include support for providing Performance Monitor information. This support consists of the perfmon module that your application uses to define the counters and provide the counter data. A DLL named perfmondata.dll is supplied as the Performance Monitor DLL; this is the DLL that the Performance Monitor loads to acquire your performance data.

Installing Performance Monitor Information

As mentioned previously, it's necessary to register with the Performance Monitor when your application is installed. To install Performance Monitor information, you must provide the following information:

?The name of a C header (.h) file providing information about the counters

?The name of a Windows INI file providing the definitions of the counters

?The name of the DLL providing Performance Monitor information

This information is necessary only at installation time: the .h and .ini files aren't required at runtime. The Performance Monitor APIs are very much C and C++ focussed, hence the requirement for the .h file. Often, this .h file would be included in the C/C++ project, but as we are using Python, we need to author a .h file that's used only for the installation process.

The header file contains #define statements for each counter. Each #define lists the offset of the specific counter in two-byte multiples. The offset zero defines the object itself.

For our sample, we provide two counters, connections per second and the bytes received per second. Thus, the header file will have three entries: the object at offset 0, the first counter at offset 2, and the last counter at offset 4.

For the sample application, use the .h file in the following code:

// PipeService2_install.h
// File used at installation time to install
// Performance Monitor Counters for our sample service.
//
// All these list the counters in order.
// Important that you add these in this order in
// the Python code.
#define SERVICE 0
#define CONNECTIONS 2 // Number of connections
#define BYTES 4 // Number of bytes received.

You also need a Windows INI file that defines the counters themselves. This INI file has three required sections: the [info] section defines the name of the appli-

cation, the [languages] section defines the list of languages in which you are providing descriptions, and the [text] section that describes the counters themselves. The INI file for the sample service looks like the following code:

; PipeService2_install.ini
; File used at installation time to install
; Performance Monitor Counters for our sample service.
;
[info]
; drivername MUST be the same as the service name
drivername=PythonPipeService
symbolfile=PipeService2_install.h
[languages]
009=English
[text]
SERVICE_009_NAME=Python Demo Service
SERVICE_009_HELP=Shows performance statistics for the sample Python service
CONNECTIONS_009_NAME=Number of connections per second.
CONNECTIONS_009_HELP=Indicates the load on the service.
BYTES_009_NAME=Bytes read/sec
BYTES_009_HELP=Number of bytes read per second by the service.

Note that the text section provides the names and descriptions for both the object itself (Python Demo Service) and the counters. Also note that the drivername entry must be the same as the service.

The final piece of the puzzle is the name of the DLL used to provide the Performance Monitor data. As described previously, a generic DLL is provided with the Win32 Extensions for this purpose. This file can be found at Python\Win32\perfmondata.dll.

Now we have all the pieces, but how do we install the data? It shouldn't be a surprise to hear that Python makes this much easier for us. As Performance Monitor information is a common requirement for services, the Python service installation procedure supports additional parameters for this purpose.

When installing a Python service, the command-line options are --perfmonini (names the INI file) and optionally, --perfmondll (names the DLL that provides the data).

Let's reinstall our service. Because the service uses the default DLL, you need to specify only the name of the .ini file. Use the command:

C:\Scripts> python.exe PipeService2.py --perfmonini=PipeService2_install.ini
install
Installing service PythonPipeService to Python class C:\Scripts\PipeService2.
PipeService
Changing service configuration
Service updated

The last message is ''service updated'' rather than "service installed." This is because the installation procedure realizes the service is already installed and simply updates the service configuration. When you remove the service later, the Performance Monitor information is automatically uninstalled.

As mentioned previously, the .h and .ini files are needed only for this one-time installation process. If you wish, you can remove these files now.

Defining the Counters

Now that you've successfully registered your service with the Performance Monitor, you need to define the counters. Then you can finally update the counters while the service is running.

The process for defining the counters is:

1. Create each counter and set its attributes.

2. Create the Performance Monitor object and add the counters to it.

3. Create a Performance Monitor manager, passing the objects.

To do this, add the following two methods to your service class:

def InitPerfMon(self):
        # Magic numbers (2 and 4) must match header and ini file used
        # at install - could lookup ini, but then Id need it a runtime
        # A counter for number of connections per second.
        self.counterConnections=perfmon.CounterDefinition(2)
        # We arent expecting many, so we set the scale high (ie, x10)
        # Note the scale is 10ÙDefaultScale = ie, to get 10, we use 1!
        self.counterConnections.DefaultScale = 1
        # Now a counter for the number of bytes received per second.
        self.counterBytes=perfmon.CounterDefinition(4)
        self.counterBytes.DefaultScale = 0
        # Now register our counters with the performance monitor
        perfObjectType = perfmon.ObjectType(
                          (self.counterConnections, self.counterBytes) )
        self.perfManager = perfmon.PerfMonManager(
                        self._svc_name_,
                        (perfObjectType,),
                        "perfmondata")
    def TermPerfMon(self):
        self.perfManager.Close()
        self.perfManager = None

Note the magic numbers passed to the perfmon.CounterDefinition() calls. These must be the offsets as defined in the .h file used at installation. The

perfmon.PerfMonManager() call also warrants some extra comments. The first parameter is the drivername as specified in the .ini file. As mentioned previously, this must be the name of the service. The last parameter is the name of the memory-mapped file that communicates the data between the service and the Performance Monitor DLL. This must be the base name of the DLL providing Performance Monitor information. The standard DLL is named perfmondata.dll, so you pass perfmondata. If you chose to rename the DLL for your own application, you must reflect the new name here and also in the installation process.

The DefaultScale property of the counter is not intuitive. This property specifies the power of 10 by which to scale a chart line (i.e., the base 10 log of the scale). For example, for a default scale of 1, you use 0. For a default scale of 10, use 1; to get a scale of 100, use 2, etc.

Updating the Counters

Now that you've defined the counters in your code, all that remains is to update the counters with your data. This is a simple process. The counter object has a method Increment() taking a single optional parameter (defaulting to 1).

Change the SvcDoRun() method, so that just before you process the client connection, you execute the following code:

# But first, increment our Performance Monitor data
self.counterConnections.Increment()

This increments the counter by 1, which is correct for your connections counter. Then, just after you write the response to the client, execute:

# Update our performance monitor "bytes read" counter
self.counterBytes.Increment(len(data))

In this case, you increment the bytes counter by the length of the data received. And that's it: your service is now providing Performance Monitor information.

Viewing Performance Monitor Data

All that remains is to view the data the service provides. The service must be running before the data is available, so let's start the new service:

C:\Scripts> python PipeService2.py debug
Debugging service PythonPipeService
Info 0x40001002 - The PythonPipeService service has started.

Now you can start the Windows NT Performance Monitor, which can be found under the Administrative Tools on the Star menu. You are presented with an empty Performance Monitor that looks something like Figure 18-4.

Now select Edit ® Add to Chart. In the dialog displayed, select the Object drop-down and scroll to Python Demo service as shown in Figure 18-5. Select and add

		Figure 18-4. The Windows NT Performance Monitor after starting

		Figure 18-5. "Add to Chart" dialog showing our counters

the two counters, and you can now start your service client and repeatedly send some data. If all goes well, your Performance Monitor display should look something like Figure 18-6.

		Figure 18-6. Collecting performance data for the sample service

A Final Service

We have made a number of changes to the sample service since first presented. Here's the full source code:

# PipeService2.py
#
# A sample demonstrating a service which uses a
# named-pipe to accept client connections,
# and writes data to the event log.
import win32serviceutil
import win32service
import win32event
import win32pipe
import win32file
import pywintypes
import winerror
import perfmon
import os
class PipeService(win32serviceutil.ServiceFramework):
    _svc_name_= "PythonPipeService"
    _svc_display_name_ = "A sample Python service using named pipes"
    def __init__(self, args):
        win32serviceutil.ServiceFramework.__init__(self, args)
        # Create an event which we will use to wait on.

    # The "service stop" request will set this event.
    self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
    # We need to use overlapped IO for this, so we dont block when
    # waiting for a client to connect. This is the only effective way
    # to handle either a client connection, or a service stop request.
    self.overlapped = pywintypes.OVERLAPPED()
    # And create an event to be used in the OVERLAPPED object.
    self.overlapped.hEvent = win32event.CreateEvent(None,0,0,None)
    # Finally initialize our Performance Monitor counters
    self.InitPerfMon()
def InitPerfMon(self):
    # Magic numbers (2 and 4) must match header and ini file used
    # at install - could lookup ini, but then Id need it a runtime
    # A counter for number of connections per second.
    self.counterConnections=perfmon.CounterDefinition(2)
    # We arent expecting many, so we set the scale high (ie, x10)
    # Note the scale is 10ÙDefaultScale = ie, to get 10, we use 1!
    self.counterConnections.DefaultScale = 1
    # Now a counter for the number of bytes received per second.
    self.counterBytes=perfmon.CounterDefinition(4)
    # A scale of 1 is fine for this counter.
    self.counterBytes.DefaultScale = 0
    # Now register our counters with the performance monitor
    perfObjectType = perfmon.ObjectType( (self.counterConnections,
                                         self.counterBytes) )
    self.perfManager = perfmon.PerfMonManager(
                    self._svc_name_,
                    (perfObjectType,),
                    "perfmondata")
def TermPerfMon(self):
    self.perfManager.Close()
    self.perfManager = None
def SvcStop(self):
    # Before we do anything, tell the SCM we are starting the stop process.
    self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
    # And set my event.
    win32event.SetEvent(self.hWaitStop)
def SvcDoRun(self):
    # Log a "started" message to the event log.
    import servicemanager
    servicemanager.LogMsg(
            servicemanager.EVENTLOG_INFORMATION_TYPE,
            servicemanager.PYS_SERVICE_STARTED,
            (self._svc_name_, ' '))
    # We create our named pipe.
    pipeName = "\\\\.\\pipe\\PyPipeService"

openMode = win32pipe.PIPE_ACCESS_DUPLEX | win32file.FILE_FLAG_OVERLAPPED
pipeMode = win32pipe.PIPE_TYPE_MESSAGE
# When running as a service, we must use special security for the pipe
sa = pywintypes.SECURITY_ATTRIBUTES()
# Say we do have a DACL, and it is empty
# (ie, allow full access!)
sa.SetSecurityDescriptorDacl ( 1, None, 0 )
pipeHandle = win32pipe.CreateNamedPipe(pipeName,
    openMode,
    pipeMode,
    win32pipe.PIPE_UNLIMITED_INSTANCES,
    0, 0, 6000, # default buffers, and 6 second timeout.
    sa)
# Loop accepting and processing connections
while 1:
    try:
        hr = win32pipe.ConnectNamedPipe(pipeHandle, self.overlapped)
    except error, details:
        print "Error connecting pipe!", details
        pipeHandle.Close()
        break
    if hr==winerror.ERROR_PIPE_CONNECTED:
        # Client is fast, and already connected - signal event
        win32event.SetEvent(self.overlapped.hEvent)
    # Wait for either a connection, or a service stop request.
    timeout = win32event.INFINITE
    waitHandles = self.hWaitStop, self.overlapped.hEvent
    rc = win32event.WaitForMultipleObjects(waitHandles, 0, timeout)
    if rc==win32event.WAIT_OBJECT_0:
        # Stop event
        break
    else:
        # Pipe event - read the data, and write it back.
        # But first, increment our Performance Monitor data
        self.counterConnections.Increment()
        # (We only handle a max of 255 characters for this sample)
        try:
        hr, data = win32file.ReadFile(pipeHandle, 256)
        win32file.WriteFile(pipeHandle, "You sent me:" + data)
        # And disconnect from the client.
        win32pipe.DisconnectNamedPipe(pipeHandle)
        # Update our performance monitor "bytes read" counter
        self.counterBytes.Increment(len(data))
        # Log a message to the event log indicating what we did.
        message = "Processed %d bytes for client connection" % \
                 len(data)
        servicemanager.LogInfoMsg(message)
    except win32file.error:
        # Client disconnected without sending data

                      # or before reading the response.
                      # Thats OK - just get the next connection
                      continue
          # cleanup our PerfMon counters.
          self.TermPerfMon()
          # Now log a "service stopped" message
          servicemanager.LogMsg(
                  servicemanager.EVENTLOG_INFORMATION_TYPE,
                  servicemanager.PYS_SERVICE_STOPPED,
                  (self._svc_name_, ' '))
if __name__=='__main__':
    win32serviceutil.HandleCommandLine(PipeService)

Conclusion

In this chapter, we have covered some advanced capabilities of Python and Windows NT Services. We discussed the key concepts behind:

?The Windows NT Service Control Manager

?The Windows NT Event Log

?The Windows NT Performance Monitor

We have presented three services written completely in Python, ranging from the trivial, to a complete service implementation that writes key events to the Event Log, provides core performance statistics via the Performance Monitor, and uses named pipes to communicate with clients.

---

Back