Using NativeProcess in AIR 2 for screencaps

• Download installable AIR file
  (requires AIR 2 Beta 2 Runtime + .NET v3.5 on Windows)

One way of learning how to use a thing is to proceed directly to trying to abuse it. My aim here was to get familiar with AIR 2's new capability of interacting with external processes' standard streams.

To that end, this AIR application takes in a fairly continuous stream of uncompressed binary image data from a native process' standard output. My hope was to get real-time screen capture updates into AIR at a decent frame rate.

The native process

For my "native process," I created a Windows .NET application written in C#. When asked, it takes a screen capture of the desktop and sends the image data through the process' standard output. For various reasons, I chose to set it up as a Windows form rather than as a plain vanilla console or as a real system service. The relevant C# code is only a few dozen lines long. One interesting function there is bitmapToBytes(), which converts a bitmap into a byte array in the correct format for Flash to be able to apply it directly to a BitmapData without any further (costly) conversion.

The AIR application

On the "client-side," the Flash AIR application has little to do. It simply sends a command to the native process, telling it to take a screen grab, along with image dimension and offset parameters. Because the client takes in the resulting stream of data in packets whose length should not be assumed to be of any given size, it needs to know where in the data a new image starts and where it ends. So it checks the stream for an arbitrary 4-byte start code as well as a termination code. When the transfer is complete, it just applies the resulting data, collected over time into a ByteArray, using BitmapData.setPixels(). It then requests a new image, and so on.

Performance considerations

On a mid-range quad-core PC using 1280x1024 images (5MB per uncompressed frame), I'm getting about 8 frames per second, or 125ms per frame. The time it takes for the 5MB image to transfer through the standard stream is about 65ms, or 50% of the total time for one iteration; without setting up an honest benchmark test, it'd be unfair and/or impolitic to declare that AIR represents a bottleneck here, so ... yeah.

The AIR app eats up about 2/3 of one CPU core using the runtime's main application thread. The remaining 1/3 represents the amount of time the AIR app has to wait while the native process constructs the screengrab. If the app were re-architected in such a way that the native process always has a new screencap queued up and ready to send - leading to little or no idle time between request and response - the AIR app would easily soak a full CPU core. Since the AIR app is doing little of significance besides reading the process's standard stream and collecting that info into a byte array, it appears that merely handling large amounts of stdout data is quite costly.

Of course, AIR 2 isn't even final yet. Nevertheless, I thought these preliminary findings were interesting. (This build targets Beta 2, which is the most recent public beta of AIR 2; compiling and running it against the most recent non-public beta of the SDK and runtime dated April 9, 2010 gave the same results.)

Source files

Setting up a native process for use with AIR 2 is well documented so I didn't bother commenting on that here. However, the included source is I believe a good starting point for adapting to whatever native process craziness you may be cooking up...

For another implementation of AIR 2 screen capture, see this post by Mihai Corlan, also with source, using Java. There are also a few other related examples out there as well.

• air
• dotnet
• flash
• sourcecode