Back to top.
Server
The server listens for new connections. Upon hearing a new connection,
it checks the access control list (ACL) and number of current users
to make sure that the client
is allowed to access the server. If not, it sends an error string,
explaining the reason the client was rejected.
If the client is accepted, the server starts listening for new commands
from the client.
It must first receive a Src command, so that future
image related commands will make sense. After receiving the Src
command, it caches the src string for future use.
After sending the Src command, the client should send a GetImage command. The server responds to this command by sending back the image in PNG format. The server also stores the raw image data for later image differencing.
Once the client has displayed the initial image, it will request an update by sending the GetUpdate command to the server. The server then takes a new snapshot of the VI, and compares the new image to the old image. It creates a list of pixels that have changed, and the new color values for those pixels. Normally it then sends this data to the client, optionally compressing it. Before sending the data, the server first compares the size of the update data to the size of the initial PNG image. If the size of the update is greater than the size of the PNG, the server chooses to send the whole image over in the form of the PNG, rather than to send the update information. This ensures that the least amout of data possible is always sent.
Upon receiving the image update (be it pixel updates or the whole PNG), the client processes the information, and then sends another GetUpdate command. This process continues until the client is shut down, at which point it sends a Stop command to the server, and closes the network connection.
If the server does not receive a command within a certain time limit, it checks to see if the communication timeout has expired, and if so, closes the connection.
Back to top.
ActionImageUpdater
Client
The ActionImageUpdater client inherits all the behavior of the
ImageUpdater client. Then, in adds to the behavior. In addition
to establishing a connection with the ImageUpdater server, it also
establishes a connection with the ActionImageUpdater server. Unless
the server rejects the connection, the client then sends the
Src command to request a specified VI with which to
interact. If the server approves, the client then is enabled to send
action commands. Keyboard and mouse events directed to the client are
processed.
If a keyboard event is received, the client determines if the event corresponds to an ASCII character. If it does, it sends the KeyType command, the character, and any modifier keys (shift, control, etc.) to the server. If it is not an ASCII character (such as an F key), the client converts the key code from its Java virtual key code to the equivalent Windows virtual key code, and sends the KeyUp command, along with the key code and any modifiers.
When the client detects a mouse down event, it does not immediately inform the server. Rather, it begins recording mouse events (moves) until it detects a mouseup event. At this point, it sends the MouseAction command, followed by all of the events. This is because the time latencies involved with the network communication are great enough that real time visual feedback will not happen anyway.
When the client is stopped, it sends the Stop command to the server to inform the server that it is terminating its connection.
Back to top.
Server
The server begins by listening for connections from clients. Upon
receiving a connection, the server first checks the ACL to make sure
that the client is allowed to interact with the server. If it is,
the server then expects the client to send the Src command to request
the target VI for the actions. The server checks to make sure that the
request VI is not already in use by another client on the assumption
that only one client should be acting on the VI at a time. It then sends
back a string informing the client of any access error, or that its
connection has been established.
Following a successful connection, the server waits for commands from the client. Upon the receipt of a KeyType or a KeyUp event, the server processes the information to determine which ASCII character (KeyType) or which virtual key code (KeyUp) it should post to the VI's window. After processing, it uses the Windows API calls to post the appropriate character message (either WM_CHAR or WM_KEYDOWN and WM_KEYUP) to inform the window that a certain message has been posted.
If the server receives a MouseAction command, it first processes all of the mouse events that make up the mouse action. It then installs a Windows message hook to intercept all messages going to the target VI. After the hook is installed, the server posts all of the mouse action commands to the target VI. Each message posted is tagged so that the hook can recognize it. The hook only pays attention to mouse messages. Any mouse message that is posted to the window while the hook is installed is inspected by the hook. If it has the tag, the tag is removed, and the message is passed on to the VI. If it does not have the tag, then the message came from the real cursor on the server machine. We do not want these mouse messages to interfere with action that we are simulating, so these messages are thrown away. Once the hook has passed all of the tagged messages to the target VI, it uninstalls itself.
When the server receives the Stop command, it closes the network connection.
As with the ImageUpdater server, if the server does not receive a command within a certain time limit, it checks to see if the communication timeout has expired, and if so, closes the connection.