Scientists are not programmers. Repeat that after me: scientists are not programmers. It’s not their fault; it’s just a lack of proper training.  If you are implementing some algorithm given you by a scientist, it’s important to know this and account for it.

Certain algorithms are not direct – most often for some process which is not easily reversible.  For example, I was given the task of implementing a way of finding the Wet-Bulb temperature, given the Dewpoint temperature, the Dry-Bulb temperature, and the Barometric Pressure.  Accompanying this task was some code, written by a scientist, in some form of BASIC.

To accomplish this, they started with an estimate (the DewPoint Temp) and worked forward, using the known equations to convert wet-bulb temp into dewpoint temp, then compared that result to the known dewpoint (Tdew).  If the result was less than the known dewpoint, they added a constant 0.05 degrees to the estimate, and tried again. When the result exceeded the dewpoint, they called it good and returned the latest estimate as the final answer.

Scientists are not programmers. If you asked them about this, they will say that it gets the right answer.  If you ask them how they came to choose 0.05 as the step size, after the blank stare (while they think about it), you will get an answer something like “Well, that’s the tolerance I want”. If you really press them, they will come up with “Well, any smaller and it’ll take too long – any larger and it’ll not be correct enough”, which is exactly true. That step size is somebody’s wild guess.

Being the obsessive speed freak that I am, I figured a better way.  What the scientist didn’t realize, is that you don’t have to have a constant step size.  With a modicum of further effort, you can adjust the step size dynamically, and get to the final answer much more quickly.

Simply start with a relatively large positive step, and do your estimates as before.  Afterward, make a decision – if you haven’t exceeded your target, step again in the same direction.  If you exceeded the target, don’t simply quit and call it good, REDUCE and REVERSE your step size and go again.  Now you’re heading negative. When you go BELOW your target, REDUCE and REVERSE your step size. Repeat this until the absolute value of your step size is below your tolerance.

In certain cases, this will take LONGER, but in the vast majority of cases where a fine tolerance is needed, this will get a more accurate answer in FEWER iterations.

You of course need to check things out and match your particular case. Use an iteration counter. You always want to reduce your step size when you reverse it: a factor of -1 would never converge and a factor of near -1 would converge slowly. But a factor of -0.01 would reduce your tolerance.  Best to use a factor of -0.1 to -0.4.

I have seen reductions of 100:1 in iteration counts between the original method and this improved search.  In cases where it was worse, it was around 15 vs. 10 iterations, in cases where it was better it was around  30 vs. 500 iterations.

Use your common sense, and don’t hold it against them.  Scientists are not programmers.

If you read the post about en masse operations, you might remember that I pointed out that you should know what is happening behind the scenes. Here is a particular case where what looks like simpler code actually takes longer to execute.  If you don’t take the time to think about what is actually going on, then you might be fooled.

Consider a pair of signals, each around 12000 samples. Regulations state that I am allowed to drop (delete) certain samples from those signals before performing statistical operations on them.  The number of points to be dropped might be 2-10%, or up  to 1200 of the points. I have the indexes to be dropped in a third array. For graphing purposes, I need to keep the dropped points in separate arrays.

Now every programmer worth his salt has fallen into the trap of deleting elements 3, 5, and 8 from an array: If you try the straightforward way, you find out that after you delete element 3, that element 5 is not in the same place it was before!  So  you either have to delete element 8 BEFORE you delete element 5 and then 3, or you have to delete element (3-0), then element (5-1), and then element (8-2).

Having fallen into that pothole my share of times many years ago, I avoided it this time by doing the reversing trick: My list of points to drop was known to be in ascending order, so I reversed it, and then did the deletions.  Because I needed the deleted points in proper order, I had to reverse those after the deletion.  Here’s the code:

That worked fine for some time, but while revisiting this code, it occurred to me that it might be faster to manipulate the index while deleting, and avoid the reversals and speed things up.  Here’s the code:

That’s certainly simpler, right?  As one should always do, I applied a Timing Measurement to it. And I was surprised.  I created two arrays of 12000 numbers and an array of 1200 random (0..11999) indexes.  It was consistently 5-6% MORE TIME this simpler way.

But if you stop and think about what’s going on, the reason is clear.  Suppose your signal array contains [0, 1, 2, 3, 4, 5] and you want to delete elements [1, 3, 4 ]

Using method A you reverse the list to get [4, 3, 1 ].
You delete element 4.  {that moves element 5 down – 1 move}
You delete element 3.  {that moves element 5 down – 1 more move}
You delete element 1. {that moves elements 2, 5 down – 2 more moves}
That’s 4 moves that were made in the shuffling process.

Now consider the “simpler” method:
You delete element [1-0].  {that moves elements 2,3,4,5 down = 4 moves }
You delete element [3-1].  {that moves elements 4,5 down = 2 moves}
You delete element [4-2].  {that moves element 5 down = 1 move }
That’s a total of SEVEN moves that were made.

So even though we eliminated three REVERSAL operations, we actually take LONGER because we are doing more work.  The increased amount of data-shuffling was enough to overcome the benefit of removing the reversals.

This was done using a random list of indexes to drop; I’d bet that there are possible scenarios where this wouldn’t hold true (for example if the points to drop were few, and confined to the end of the signal), but given that neither of those will be true in my case, I’m sticking with the original plan – on average it will be faster.

But don’t assume that fewer operations on the diagram means less work !

As mentioned earlier, a compiled LabVIEW application behaves similarly to the development system when terminating.  Namely, it leaves the main window on the screen, waiting for you to close it.  That’s handy in the DevSys, because you usually want to work some more on the program after quitting.

But in an executable, it’s not so good, because the user doesn’t understand why the window hangs around.

The earlier article offered a way to have it both ways by simply detecting whether or not you were running with the main VI from an LLB, or something else, and performing a QUIT LABVIEW if it was something else.

With the advent of LabVIEW 2009, the scheme of detecting whether you were in an LLB or not was broken, because LV2009 started putting VIs into an EXE using the folder structure that they came from.  Before 2009, an EXE was a container for ALL VIs in the program, regardless of their folder structure on disk.  It was like having one folder.

Using THIS VI’s PATH would point to one of those VIs, stripping it ONCE would point to the container, and stripping it TWICE would refer to the containing folder.

With LV2009 and later, we can no longer use that logic.  What we do instead is to examine the path to MAIN:

• If we find an “.EXE”  in it, then we are in an executable, and we strip twice to get the containing folder
• else if we find “.LLB” in it, we are in a library, and we strip twice (from the point of the LLB) to get the containing folder
• else if we find “.VI” in it, we are in a stand-alone VI and strip ONCE to get to the containing folder

The attached VI is a replacement for the ROOT FOLDER vi mentioned in the earlier article, and is in LV 2009 format  (works in LV2010, too).

Use it when you’re ready to quit – see this snippet:

Click to download the Root Folder.vi .

By using the NOT IN LIBRARY signal as an input for QUIT LABVIEW, you can run the same code unmodified in an app, or in the DevSys, and it does the right thing either way.

Any version of NI-DAQ and the Measurement and Automation Explorer (MAX) released recently has provisions for “simulated” devices.  You choose which devices you want, and then NI-DAQ will pretend those devices are actually installed on your system, any calls to DAQ functions concerning that device will succeed (or fail) just as if a real device was installed.

This lets you simulate a client’s setup without having their hardware shipped to you and do most (if not all) of the programming on your own terms without being at their site.  The data produced is, of course, simulated data.  For an analog input channel it’s a sine wave, for a digital port, it’s a counting pattern.  It’s enough for you to tell if your software is working correctly with NI-DAQ.

With their hardware simulated on your machine, you can handle the basic communication part to get data in and out. Then you can install conditional-compilation pieces to substitute data more realistic for your particular situation if you need to.

You can be reasonably confident that the DAQ part of a program you develop this way will work on the real hardware, the same as it did on your simulated hardware.  Of course, for any extreme cases (high sample rate, high channel count), the simulation will be less exact, but it’s a useful feature to develop faster with fewer headaches.

“Fragile” code is code that breaks in one place because of changes you make in some other place. It’s most aggravating when you’re due to ship a new version tomorrow and you need to make one last tweak at 11:30 PM, or your client is looking over your shoulder and this little “harmless” change shows up as a smoldering heap during the demo.

In this case, “break” doesn’t ONLY mean “broken arrow” , or uncompilable code (at least you can chase those down easily enough). Here, “break” also means “operates incorrectly” or “completely wrecks itself like it never did before” or somewhere in between.

These sorts of breaks come from unrecognized dependencies, and they’re all too easy to make: the header size has been 3 for months and months now, so when you add a new function that needs it, it’s easy to stick in a constant 3 and be done with it.

DON’T DO IT.

If you’re an old-hand bit-banging cycle counter like me, it’s easy to think of saving a few cycles and adding up the bytes in this cluster, and using a constant of 53 when you need the size of it.

DON’T DO IT.

The problem is, or course, then when (not if, but when) these things change, then you will have to track down ALL the instances where you use this number and change them.  Not a good plan.  The compiler won’t complain – the code is still valid.  But reading thee bytes when you should be reading four will not get you where you want to go.

One tip to solving this is to reduce the number of places that you use such numbers. Focus such procedures into a single VI if you can. But the real key to solving this is to recognize these things when you originate them.

Example #1

Here we need to receive a packet header, consisting of a cluster of a U8 enum command and a U16 integer.  It’s easy enough to count up to three bytes, and it would be easy to plop down a 3 constant.  However, it is safer to use a constant of the header’s typedef and calculate its size in code.  This might go against your instincts (it does mine), but in fact the extra time taken (to flatten into string and get string length) is trivial (measure it yourself if you have doubts). On top of a TCP READ operation, this burden is truly insignificant.

And the benefit is that when the integer needs to become an U32 or the command must become a U16, here’s one less thing YOU have to worry about.  Since the constant here is a TYPEDEF, and since you’re calculating the size every time, then it will keep on working.

Example #2

This example is similar – sending a packet header plus a payload thru a connection. Even if your payload is always the same size, it’s better to calculate it than to use a constant.  With any luck at all, the STRING LENGTH operation will get the same answer every time. and if you do change it at some point, then this code won’t break.

Some development environments have a concept called “watching”, where you choose a variable to watch and you see a continuous display of that variable in some window.  This is very useful during debugging, as you can step through your program and find out where this variable is being changed.

LabVIEW has no such built-in feature, but it doesn’t really need one.  You can construct your own watch windows, have them run independently of your main code and accomplish the same thing.

Simply make a new VI with a WHILE loop and a STOP button.  Add a WAIT for 200 mSec (or something) inside it (so you don’t hog the CPU).  Each time thru the loop, grab your watch variable, process it, and display it.

The “processing” can be unbundling a single item from a complicated cluster, or picking an element out of an array, or anything you need to display the item in question.  Perhaps you need to call a VI to get it. Perhaps you need to query an I/O port, or a TCP instrument. Whatever you need to do to watch your troublesome variable.

SUGGESTION:  When you’re done with it, save it in a folder called “Miscellaneous Stuff” or something, so you can get at it easy next time.  There will be a next time.

A question came up on the LabVIEW forum the other day about multiple connections, and how hard it was to have two connections transmitting at two different rates.  This surprised me a bit, because I have been doing just that for quite a few years.  The allegation was made also that TCP requires a minimum packet size of 32 bytes. That is also a surprise since I have been doing things contrary to that “rule” for quite a few years.

So, in an attempt to clarify things above and beyond the Beginner’s Guide to TCP/IP, I present this example with a CLIENT and a SERVER.

The SERVER should be run first and listens for four connections on four consecutive ports. It has four data generators, running at different rates.  When data is available, it transmits it over the connection  if there is one, or listens for one if there’s not.

The SERVER uses a re-entrant VI so that the same code can be executing in four instances at one time.  The four instances are given four different intervals. The WAITs do not conflict because of this reentrancy.

The CLIENT initiates four connections to these same ports, and waits for data in four loops.  I did not use reentrant VIs here, because of the connection to the charts.  Although you could pass a reference to the chart to four reentrant subVIs, I chose not to, thinking that the speed would be better with the direct approach.

Given that this is based on the mSec Timer in LabVIEW, I wouldn’t vouch for it’s accuracy as you approach 1000 Hz. But the basic techniques are sound and will work beyond that frequency.

Click here for a downloadable example of four-channel client server communications.

Just in case you thought I was kidding in the article on en masse operations, I decided to offer some proof of the speed advantages they can give you.

I used the Timing Template vi to measure the time it takes to multiply an array of DBLs by two, both with a loop, and without.  I set up the timing VI to create an array of 1000 random numbers, and then time the multiply operation.

First, the loop method, where you auto-index every value out of the array, multiply it, and auto-index it back in:

As you can see, this took 7.47 uSec per loop.  Not all that shabby.  But just removing the loop lets the en masse operation do it:

Holy Speed Demon, Batman!  That’s 0.77 uSec or about ONE TENTH the time!

Now, I’m not guaranteeing that all such operations will save you that much time, but if you have a chance to use them, then you should!

It’s easier on you and easier on the hardware!

LabVIEW charts, out of the box, don’t lend themselves to displaying the actual time of day.  By default they give you 1024 history points and a visible scale of 0-100 so what you see is in terms of sample numbers, having no relation to actual time. This is reasonable, given that the scale is adjustable in so many ways; there’s no way to guess what you want, so it’s up to you to tailor it to fit what you want.

Sometimes it’s useful to display the time of day on your charts, and keep that axis correctly updated as data flows in. For example:

As you can see, the leading edge of the plots coincides with the current time on the clock  (see here for how to do the clock indicator).   As data keeps coming in to this chart, the leading edge will reach the right side and the chart (and the X-axis scale) will scroll to the left, thus always matching the time of day.

Out of the box, a LabVIEW chart has a history of 1024 samples, and an X scale of 0-100 samples.  This scale has nothing to do with time, so if you want it to display the time, then you have to do several things to make it work.  If you have a CLEAR CHART function, triggered by a button or something, then attach this code to that action, otherwise just do it once when you start the data flowing  (Ideally, you should do this as you feed the first sample to the chart, but the few mSec difference that makes probably won’t matter).

First, you clear the history.  That makes sure the OFFSET number is operating from an empty buffer (Time 0).  If the history buffer contains data, then the OFFSET applies to the end of it, and that’s not what you want.  The constant has to be an empty array of the same type of data as you feed to the chart (cluster of two DBLs in this case).

Next you set the XSCALE.OFFSET to set the OFFSET number to the current time.

Next you set the XSCALE.MULTIPLIER to the time (in sec) between samples that you display.

Next you set the XSCALE.MAXIMUM to now + the duration (in seconds) that you want the chart to show.

Do them in that order, or it could get confusing.  You also can set the CHART HISTORY LENGTH to match the duration / the display period.  For one minute at 2 Hz, that would be 120 samples.  You only need to do that step once – it’s a property that is not programmable.

Enjoy.

]]> http://culverson.com/keeping-your-charts-up-to-date/feed/ 2 http://culverson.com/an-improved-analog-clock/ http://culverson.com/an-improved-analog-clock/#comments Wed, 19 Aug 2009 15:58:35 +0000 Steve http://jimdugan.com/culverson/?p=53 Sometimes all that digital stuff is just too bland.

A bug undocumented feature of the original analog clock was that the markers on the scale were at intervals of 1.25 seconds, a consequence of LabVIEW preferring to use 4 intervals per major tick, when we silly humans use 5.  As a result, it looked a bit odd.  As I said then, if we’re going for an analog, then let’s go for the analog.

Attempts to coerce LabVIEW into making the scale the way we wanted were not successful; it has a long habit of thinking that four is a nice number and five is just an odd number, so I could not make it work.

So how do we improve it?  Simply disregard the built-in scale and substitute a picture.  After all, we all know where the numbers are on a clock face, if we’ve lined up the 0 and the 12 on our LabVIEW scales at the top of the circle, then all else has to fall into place.

Click here to download the example LLB file, in LV 8.0 format. It has the same math VI as before (read the previous post for details), just the clock face is different.

Enjoy.