Scientists are not programmers. Repeat that after me: scientists are not programmers. It’s not their fault; it’s just a lack of proper training.  If you are implementing some algorithm given you by a scientist, it’s important to know this and account for it.

Certain algorithms are not direct – most often for some process which is not easily reversible.  For example, I was given the task of implementing a way of finding the Wet-Bulb temperature, given the Dewpoint temperature, the Dry-Bulb temperature, and the Barometric Pressure.  Accompanying this task was some code, written by a scientist, in some form of BASIC.

To accomplish this, they started with an estimate (the DewPoint Temp) and worked forward, using the known equations to convert wet-bulb temp into dewpoint temp, then compared that result to the known dewpoint (Tdew).  If the result was less than the known dewpoint, they added a constant 0.05 degrees to the estimate, and tried again. When the result exceeded the dewpoint, they called it good and returned the latest estimate as the final answer.

Scientists are not programmers. If you asked them about this, they will say that it gets the right answer.  If you ask them how they came to choose 0.05 as the step size, after the blank stare (while they think about it), you will get an answer something like “Well, that’s the tolerance I want”. If you really press them, they will come up with “Well, any smaller and it’ll take too long – any larger and it’ll not be correct enough”, which is exactly true. That step size is somebody’s wild guess.

Being the obsessive speed freak that I am, I figured a better way.  What the scientist didn’t realize, is that you don’t have to have a constant step size.  With a modicum of further effort, you can adjust the step size dynamically, and get to the final answer much more quickly.

Simply start with a relatively large positive step, and do your estimates as before.  Afterward, make a decision – if you haven’t exceeded your target, step again in the same direction.  If you exceeded the target, don’t simply quit and call it good, REDUCE and REVERSE your step size and go again.  Now you’re heading negative. When you go BELOW your target, REDUCE and REVERSE your step size. Repeat this until the absolute value of your step size is below your tolerance.

In certain cases, this will take LONGER, but in the vast majority of cases where a fine tolerance is needed, this will get a more accurate answer in FEWER iterations.

You of course need to check things out and match your particular case. Use an iteration counter. You always want to reduce your step size when you reverse it: a factor of -1 would never converge and a factor of near -1 would converge slowly. But a factor of -0.01 would reduce your tolerance.  Best to use a factor of -0.1 to -0.4.

I have seen reductions of 100:1 in iteration counts between the original method and this improved search.  In cases where it was worse, it was around 15 vs. 10 iterations, in cases where it was better it was around  30 vs. 500 iterations.

Use your common sense, and don’t hold it against them.  Scientists are not programmers.

If you read the post about en masse operations, you might remember that I pointed out that you should know what is happening behind the scenes. Here is a particular case where what looks like simpler code actually takes longer to execute.  If you don’t take the time to think about what is actually going on, then you might be fooled.

Consider a pair of signals, each around 12000 samples. Regulations state that I am allowed to drop (delete) certain samples from those signals before performing statistical operations on them.  The number of points to be dropped might be 2-10%, or up  to 1200 of the points. I have the indexes to be dropped in a third array. For graphing purposes, I need to keep the dropped points in separate arrays.

Now every programmer worth his salt has fallen into the trap of deleting elements 3, 5, and 8 from an array: If you try the straightforward way, you find out that after you delete element 3, that element 5 is not in the same place it was before!  So  you either have to delete element 8 BEFORE you delete element 5 and then 3, or you have to delete element (3-0), then element (5-1), and then element (8-2).

Having fallen into that pothole my share of times many years ago, I avoided it this time by doing the reversing trick: My list of points to drop was known to be in ascending order, so I reversed it, and then did the deletions.  Because I needed the deleted points in proper order, I had to reverse those after the deletion.  Here’s the code:

That worked fine for some time, but while revisiting this code, it occurred to me that it might be faster to manipulate the index while deleting, and avoid the reversals and speed things up.  Here’s the code:

That’s certainly simpler, right?  As one should always do, I applied a Timing Measurement to it. And I was surprised.  I created two arrays of 12000 numbers and an array of 1200 random (0..11999) indexes.  It was consistently 5-6% MORE TIME this simpler way.

But if you stop and think about what’s going on, the reason is clear.  Suppose your signal array contains [0, 1, 2, 3, 4, 5] and you want to delete elements [1, 3, 4 ]

Using method A you reverse the list to get [4, 3, 1 ].
You delete element 4.  {that moves element 5 down – 1 move}
You delete element 3.  {that moves element 5 down – 1 more move}
You delete element 1. {that moves elements 2, 5 down – 2 more moves}
That’s 4 moves that were made in the shuffling process.

Now consider the “simpler” method:
You delete element [1-0].  {that moves elements 2,3,4,5 down = 4 moves }
You delete element [3-1].  {that moves elements 4,5 down = 2 moves}
You delete element [4-2].  {that moves element 5 down = 1 move }
That’s a total of SEVEN moves that were made.

So even though we eliminated three REVERSAL operations, we actually take LONGER because we are doing more work.  The increased amount of data-shuffling was enough to overcome the benefit of removing the reversals.

This was done using a random list of indexes to drop; I’d bet that there are possible scenarios where this wouldn’t hold true (for example if the points to drop were few, and confined to the end of the signal), but given that neither of those will be true in my case, I’m sticking with the original plan – on average it will be faster.

But don’t assume that fewer operations on the diagram means less work !

As mentioned earlier, a compiled LabVIEW application behaves similarly to the development system when terminating.  Namely, it leaves the main window on the screen, waiting for you to close it.  That’s handy in the DevSys, because you usually want to work some more on the program after quitting.

But in an executable, it’s not so good, because the user doesn’t understand why the window hangs around.

The earlier article offered a way to have it both ways by simply detecting whether or not you were running with the main VI from an LLB, or something else, and performing a QUIT LABVIEW if it was something else.

With the advent of LabVIEW 2009, the scheme of detecting whether you were in an LLB or not was broken, because LV2009 started putting VIs into an EXE using the folder structure that they came from.  Before 2009, an EXE was a container for ALL VIs in the program, regardless of their folder structure on disk.  It was like having one folder.

Using THIS VI’s PATH would point to one of those VIs, stripping it ONCE would point to the container, and stripping it TWICE would refer to the containing folder.

With LV2009 and later, we can no longer use that logic.  What we do instead is to examine the path to MAIN:

• If we find an “.EXE”  in it, then we are in an executable, and we strip twice to get the containing folder
• else if we find “.LLB” in it, we are in a library, and we strip twice (from the point of the LLB) to get the containing folder
• else if we find “.VI” in it, we are in a stand-alone VI and strip ONCE to get to the containing folder

The attached VI is a replacement for the ROOT FOLDER vi mentioned in the earlier article, and is in LV 2009 format  (works in LV2010, too).

Use it when you’re ready to quit – see this snippet:

Click to download the Root Folder.vi .

By using the NOT IN LIBRARY signal as an input for QUIT LABVIEW, you can run the same code unmodified in an app, or in the DevSys, and it does the right thing either way.

Any version of NI-DAQ and the Measurement and Automation Explorer (MAX) released recently has provisions for “simulated” devices.  You choose which devices you want, and then NI-DAQ will pretend those devices are actually installed on your system, any calls to DAQ functions concerning that device will succeed (or fail) just as if a real device was installed.

This lets you simulate a client’s setup without having their hardware shipped to you and do most (if not all) of the programming on your own terms without being at their site.  The data produced is, of course, simulated data.  For an analog input channel it’s a sine wave, for a digital port, it’s a counting pattern.  It’s enough for you to tell if your software is working correctly with NI-DAQ.

With their hardware simulated on your machine, you can handle the basic communication part to get data in and out. Then you can install conditional-compilation pieces to substitute data more realistic for your particular situation if you need to.

You can be reasonably confident that the DAQ part of a program you develop this way will work on the real hardware, the same as it did on your simulated hardware.  Of course, for any extreme cases (high sample rate, high channel count), the simulation will be less exact, but it’s a useful feature to develop faster with fewer headaches.

Just ran into what at first appeared to be a bug, but turned out to be proper, if misunderstood, behavior.

I have a project which records data files. When the actual recording starts, and again when it stops, I remember the time (by using a GET DATE/TIME in SECONDS function) in a TIMESTAMP variable, which is stored in the data file. There might or might not be some calibration activity after the recording has stopped.  When the DONE button is finally clicked, I record the current time using a FORMAT DATE and TIME STRING function, into another string field called “TEST TIME”

I have a viewer which examines the data files and reports various info about them. The indicator that shows the TEST TIME is on a different window from the one that shows the START TIME and END TIME.

If there’s no CAL operations done, then the TEST TIME has always been just a few seconds later than the STOP time (enough time to react to the test being done and click the DONE button), or it could be a few minutes later.

However, I recently noticed that, on a data file sent from my client to me, that the TEST TIME was almost an hour EARLIER than the STOP time.  How could that be?  Further rummaging thru other files that I had from him shows the same thing: the TEST TIME was short of an hour EARLIER than the DONE time.  If there were CAL operations done, this difference was 45-55 minutes; if not, it was a few seconds short of an hour.

I have run thousands of tests on my machine without noticing this; my client has also run nearly a thousand, and has never brought it up.  Whay is that?

There’s one piece of info missing from the above description that’s critical to figuring out what’s happening: My client is on CENTRAL time, while I am on EASTERN time.

It took a while to figure out what was really happening:  The TIMESTAMP value was being sent whole: encoded with that value somewhere must be the TIMEZONE that it was recorded in.  When the file crossed the timezone line, the value (as it was displayed to me) actually changed.  I confirmed this with my client; a given file showed a START time of 9:39:20 on his machine, but the same file showed a START time of 10:39:20 on my machine.  Of course, those are the exact same moments, just expressed in different time zones.

The TEST TIME, however was recorded as a STRING, and as such, has no mechanism for changing.

Determinism in software is the ability to ensure that any and all paths taken through the code take a consistent amount of time to execute.  Most desktop applications have no interest in this consistency because A) it doesn’t affect anything, and B) the existence of interrupts and preemptive multitasking means that you cannot do anything about it anyway.

In an embedded control application, or one running under a real-time OS however, determinism is often important because it is critical that the control outputs be changed at a consistent time with respect to the control input sampling time. On the input side, when a signal is changing at a relatively rapid rate, any error in the TIME of measurement is just as destructive to the measurement accuracy as an error in AMPLITUDE.  For applications such as PID loops, this is even more important, since derivative terms are adversely affected by timing inaccuracies.

Equally important, but not always equally recognized, is the fact that the timing of the output samples has exactly the same effect.  If your control output is not consistently timed, then loop stability is compromised.

One way to make your code deterministic is to go through it and make sure that all path lengths are identical.  For example, if your code includes an option to scale the signal to percent of full scale or not, then the proper way to make this deterministic is to compute the full-scale percentage in all cases, then decide to use the scaled or the unscaled value depending on the option setting.  This goes against the programmer’s natural instinct to do work only when required; but it’s necessary to ensure that both paths consume an equal time.

Beginners to real-time programming are rightly warned about this situation.  If you look at the larger picture, you realize that the goal of all this is to ensure that the outputs change at a consistent time relative to each other and to absolute time.  The assumption inherent in the idea of equalizing code paths is that you want the outputs to follow as soon as possible after the control inputs are sampled.

That assumption is true in some cases, but by no means all.  There is another way to achieve the same result (consistency of output timing), without the drudgery of counting cycles and finding all the places where such path decisions are made.

By building in a hardware-controlled delay, you can achieve rock-solid precision of output timing, without fretting over every single operation. One way to do this is to use the very same clock signal that drives the input timing to drive the output timing.  For example, in the following code, we:

• Create the task for the Analog Output channels
• Set the AO SAMPLE CLOCK timing parameters to the AI clock signal
• Start the task

This instructs the OUTPUT task to use the same clock as the INPUT.  When you WRITE to the output task, the values you write will stay in the output buffer until the clock ticks; at that point they will be transferred to the actual analog output.

What Does this Do for Me?

By doing this, you have freed yourself from all constraints of cycle-counting and path-equalization. Your control output will change exactly one clock tick after the inputs that produce it. No more and no less.  You have only one constraint: you must get the job done in one clock cycle. If you are running at a loop rate of 100 Hz, then you have 10 mSec to get your work done. If it takes 10 microseconds or 9.99 mSec, it makes NO difference, the output will change at the same point in time either way. You do have to ensure that it’s 10 mSec or less, but that’s a constraint you have in any case.

In most cases, where you are waiting on a clock-controlled input sample, performing some control processing with the measured values and writing control values out, the delay you are introducing is not significant.  And a precisely controlled constant delay is often more easily dealt with in a control loop than an unpredictable jitter, and easier to produce in hardware than in software.

“Fragile” code is code that breaks in one place because of changes you make in some other place. It’s most aggravating when you’re due to ship a new version tomorrow and you need to make one last tweak at 11:30 PM, or your client is looking over your shoulder and this little “harmless” change shows up as a smoldering heap during the demo.

In this case, “break” doesn’t ONLY mean “broken arrow” , or uncompilable code (at least you can chase those down easily enough). Here, “break” also means “operates incorrectly” or “completely wrecks itself like it never did before” or somewhere in between.

These sorts of breaks come from unrecognized dependencies, and they’re all too easy to make: the header size has been 3 for months and months now, so when you add a new function that needs it, it’s easy to stick in a constant 3 and be done with it.

DON’T DO IT.

If you’re an old-hand bit-banging cycle counter like me, it’s easy to think of saving a few cycles and adding up the bytes in this cluster, and using a constant of 53 when you need the size of it.

DON’T DO IT.

The problem is, or course, then when (not if, but when) these things change, then you will have to track down ALL the instances where you use this number and change them.  Not a good plan.  The compiler won’t complain – the code is still valid.  But reading thee bytes when you should be reading four will not get you where you want to go.

One tip to solving this is to reduce the number of places that you use such numbers. Focus such procedures into a single VI if you can. But the real key to solving this is to recognize these things when you originate them.

Example #1

Here we need to receive a packet header, consisting of a cluster of a U8 enum command and a U16 integer.  It’s easy enough to count up to three bytes, and it would be easy to plop down a 3 constant.  However, it is safer to use a constant of the header’s typedef and calculate its size in code.  This might go against your instincts (it does mine), but in fact the extra time taken (to flatten into string and get string length) is trivial (measure it yourself if you have doubts). On top of a TCP READ operation, this burden is truly insignificant.

And the benefit is that when the integer needs to become an U32 or the command must become a U16, here’s one less thing YOU have to worry about.  Since the constant here is a TYPEDEF, and since you’re calculating the size every time, then it will keep on working.

Example #2

This example is similar – sending a packet header plus a payload thru a connection. Even if your payload is always the same size, it’s better to calculate it than to use a constant.  With any luck at all, the STRING LENGTH operation will get the same answer every time. and if you do change it at some point, then this code won’t break.

I discussed the idea of a time-alignment scheme in the article Delays, Delays, Delays.  The idea is that signals which have a mechanical delay of some kind (gas transport time, for example) can be time-aligned with signals that have a lesser or no such delay by means of a “logical” delay line inserted into all channels.  If the sum of all channel’s physical + logical delays is a constant, then the final output is time-aligned.

Here is a test of that concept, where three signals are generated, with different phases, and then fed thru that process. The outputs, as you can see, have been aligned.

You have to know the physical delay time, however; it does not guess.  You also have to account for the fact that some data is lost at the beginning, if that’s of concern, then start your recording early enough to account for that.

Even with those caveats, this scheme allows you to shift signals to account for mechanical delays, EVEN IN REAL TIME, if you need to.

Operators of large industrial engines are able to measure the deflection of a single segment of a crankshaft (known as a web) to determine that web’s stress on the shaft. Unfortunately the operators that contracted us for this program were unable to assess the cumulative stress of multiple webs on a single shaft.

Based on a unifying equation, our program allows users to compile multiple web deflection measurements into a complete and easy to understand picture of the crank shaft’s shape and motion.

Using this program has helped many operators prevent crankshaft breakage (a costly occurrence) by focusing their maintenance dollars on the units showing the worst distortion.

MainThe main window shows a diagram that the user can manipulate to indicate throw and bearing positions, and miscellaneous input parameters.

For each throw on the crankshaft, the user enters readings of web deflection and the program calculates the path taken by the web as it rotates.

This is a graph of the bearing displacements, shown as deviations from a straight line.

By recording the bearing displacements over time, an animation can be displayed, alerting the user to possibly dangerous trends.

Using the calculated displacements, a 3-D graph can be constructed to clearly indicate physical deformation of the shaft. The user can view the deformed shaft from any angle.

Some development environments have a concept called “watching”, where you choose a variable to watch and you see a continuous display of that variable in some window.  This is very useful during debugging, as you can step through your program and find out where this variable is being changed.

LabVIEW has no such built-in feature, but it doesn’t really need one.  You can construct your own watch windows, have them run independently of your main code and accomplish the same thing.

Simply make a new VI with a WHILE loop and a STOP button.  Add a WAIT for 200 mSec (or something) inside it (so you don’t hog the CPU).  Each time thru the loop, grab your watch variable, process it, and display it.

The “processing” can be unbundling a single item from a complicated cluster, or picking an element out of an array, or anything you need to display the item in question.  Perhaps you need to call a VI to get it. Perhaps you need to query an I/O port, or a TCP instrument. Whatever you need to do to watch your troublesome variable.

SUGGESTION:  When you’re done with it, save it in a folder called “Miscellaneous Stuff” or something, so you can get at it easy next time.  There will be a next time.