En masse is a French term meaning “as a whole” or “all together”; treating a group of something as a single unit.   LabVIEW has the ability to treat arrays this way, which can greatly reduce your workload. If you come to LabVIEW from a text-based language, it’s easy to miss the capabilities that are right at your fingertips.

For example, if you need to scale a series of readings into percent of the total (a procedure called normalizing),  then you tend to think:

I need to find the total:

• I need to start with a zero sum     sum = 0.0;
• I need to loop over every element  for (int i = 0; i < nElements; i++)
• I need to add this element to the sum   sum += array[i]

Now I need to divide each entry by the total, to get the fraction of the total:

• for (int i = 0; i < nElements; i++)
• array[i] /= sum;

Now I need to multiply by 100 to get percentages:

• for (int i = 0; i < nElements; i++)
• array[i] *= 100.0;

That’s all well and good, and you could translate that literally into LabVIEW and it will get you the answer you want to see.  But that’s not the LabVIEW way of thinking.

What newcomers often fail to realize is that most primitive numeric functions (the ones with yellowish icons) will accept an array of numbers directly. This goes for basic arithmetic (add,subtract, multiply, divide), comparisons (greater than, less than, MAX/MIN), and many other operations.  It will happily multiply an array of numbers by a single scaler number, to produce an array of numbers.

This has great power to reduce the work that you do as the programmer. Consider the literal translation of the above code:

If that’s as good as it gets then why should I go with LabVIEW?

Well, it does get better.  There is a function in the numeric palette called ADD ARRAY ELEMENTS.  If we replace the entire first loop with this function, then we get to this:

Now for the en masse parts: You can replace the entire second loop with a single operation as well:

Any guesses what we can do with the third loop?    Yes, that’s right:

Now you have SO much more room to add comments about what you’re doing!

Now THIS is what makes you more productive in LabVIEW than in C; your chances for error are far less when you let the en masse operators handle the details, and you don’t even have to think about the details.

But be aware of what’s going on, however; there is no magic here.  Under the hood there is still a loop somewhere.  It’s now hidden somewhat; it’s not as obvious, but the work is still being done.  Don’t let the simplicity obscure the real processing that’s going on.

Here is an example of the normalizing function in use, from the real LabVIEW example examples\general\graphs\charts.llb\Draw Stacked Graph.vi (in LabVIEW 8.6. anyway).

This amounts to the same as our last part above.  In the example, the array given contains five elements and this is executed only once, so efficiency is not a concern.

But consider if the array was 10,000 elements. Don’t forget that the first operation is doing 10,000 divide operations, and the second is doing 10,000 multiplications.  Can you improve things?

Well, certainly! What you have to realize is that, by the associative property of numbers, (X / sum) * 100 is equal to (100 / sum) * X.  You also have to realize that 100 / sum, in this context, is a constant, and therefore needs to be calculated only once.  In effect, you are dividing by sum and multiplying by 100, but you are doing it 10,000 times!

With any luck at all, you get the same answer every time, so you only need to do it once:

THIS is why we use LabVIEW!

NOTE:  En masse is my term for this feature, it is not an official LabVIEW term.

Usually, in a data acquisition program,  all the signals you measure are “live”, meaning they represent the current conditions at the time they are sampled. However, in some cases you might have some signals which are not live, but delayed. For example, suppose you’re measuring engine operation, and you have gas analyzers sampling the exhaust airstream. These analyzers conduct the gas from the measurement point in the airflow system around the engine to the analyzer mechanism itself. This gas flows through the sampling tubing at a specific rate, and therefore arrives at the sample point at a specific time AFTER it left the main airstream. These delays might be multiple seconds in duration.  There could be other reasons for this delay, such as an echo-measuring device, or the mechanical response time of some piece of hardware.

For analysis purposes, you want to look at the engine holistically and see causes and effects. If the speed changed at a particular point in time, you want to see the CO concentration change as a consequence. But unless you do something to compensate, the change in gas concentration will lag far behind the change in speed that caused it in graphs and tables, making it difficult to judge consequences.

To arrange the signals back into time-synchronous alignment, the solution is to think of ALL signals has having TWO delay lines: one physical, and one logical.  In our example case, the physical part is the gas piping conducting the gas. The logical part is completely within the  DAS software. If EVERY channel has a physical delay time (even if it’s zero), and if EVERY channel also has a logical delay time (even if it’s zero), and if EVERY channel has the sum of the physical delay time and the logical delay time equal to a constant, then the signals coming out of the logical delay lines will be time-aligned.

Example

For example, consider three channels: Channel A records data “live”, i.e. no delay. Channel B has a delay of 3 seconds. Channel C has a delay of 5 seconds.

We want to make the entire chain (physical + logical) the same length for all channels, so we pick the longest delay (5 sec) and make that our total delay time. For channel A, which has no physical delay, we have to have a logical delay of 5 sec. For channel B, which has a physical delay of 3 sec, we add a logical delay of 2 sec for a total of 5. For channel C, which has a physical delay of 5 seconds, we have a logical delay of zero. 5+0 = 3+2 = 0+5, so every channel has the same delay, when we count both physical and logical.

The logical delay is implemented in queues with every channel having its own queue. Every sample received goes into a queue belonging to that channel; and we pull out one sample from the end of the queue to be recorded. Since one goes in and one comes out, the length of the queue never changes. The initial length of the queue corresponds to the delay time we need for a particular channel. If a queue is empty (of zero length), then the sample we put in and the sample we take out are the same sample. If the queue is of length 10, then the sample we get out was taken 10 sample times ago.

At configuration time, the queues are set up, and populated with zero values to set their length according to how much delay is required. After that, the sample process means putting a sample into a queue and taking one out for use.  Every channel operates the same when actually sampling, it’s only the configuration where they differ.

Note that if you are recording a TIMESTAMP value, then the TIMESTAMP signal requires its own queue as well, so that it comes out in alignment with the signals. Of course this means that the data does not become usable until all the zeroes have been flushed out of the queues. That will happen when the longest delay time has expired. We can judge this by putting a RECORD signal into its own queue. We don’t actually record data coming out of the queues until the RECORD signal coming out is true.

You can keep the data flowing into (and out of) these queues all the time.  When you want to start recording, set the RECORD flag TRUE. When you want to stop recording, set the RECORD flag FALSE.  Pay attention to the value out of the RECORD flag’s queue, and act appropriately. (Don’t act on the RECORD flag itself, act on the delayed RECORD flag).

This also means that when the test is over, there is still data remaining in the queues. We have to keep recording until the valuable data has worked it’s way through the queues, meaning 5 seconds after the last sample (in the example case).

In LabVIEW, there are three kinds of files:

• TEXT files. Ordinary text, stored in human-readable form, with spaces and line feeds, etc.
• BINARY files. Raw information stored as machine-readable information. A 32-bit integer is stored in 4 bytes. A double-precision number is stored in 8 bytes.
• DATALOG files. Structured data, suitable for quick transfer between a memory structure and the disk file.

I have long used Datalog files for configuration data. They offer several advantages:

• EXTREMELY simple reading/writing. No matter how complicated the data structure, you just open the file, write the structure to it, and close the file. No muss, no fuss. I have a project where the datalog file is a single record of maybe 200 k Bytes. It’s still all done with a single WRITE FILE call.
• Being binary, users are reluctant to open it in their text editors and twiddle about. You won’t get a service call to figure out that the user set the serial port to COM1.76 and the alarm level to 0 ( But it was working perfectly yesterday! ).
• Compactness. A DBL is 8 bytes. The value -12.3456 is 8 bytes, not even counting the delimiters separating it from it’s neighbors. If space is truly an issue, use SGL (4 bytes) instead of DBL.
• If your panel is set to allow a number to vary between 10 and 20, then that’s what gets stored. You don’t have to check every number in the file to see if it’s in range. Since you wrote it there, it’s correct.
• Format checking. Even if you use the standard file dialogs for choosing files (rather than a custom one), you can set it so that it will show only files of the correct type. The user has fewer wrong files to choose from, therefore the odds of a mistake are lessened.

There are a couple of disadvantages, though:

• Rigid formatting. The thing that makes it so easy to read and write, turns around and complicates things when it’s time to revise the format. If you add or remove so much as a single item, or re-arrange the order of things, then the old files are not readable anymore. You can attempt to compensate for this by adding spare fields at the start, but if you make a change to the format, you will have to make an updater which reads the old files and writes new ones in their place, or else all the old files are worthless.
• Non-portability. The tamper-resistance feature can be a disadvantage if the file must be available in other (non-LabVIEW) applications. For this reason, datalog format is best suited to files that have limited, well-defined uses.

Typically, a data-acquisition program, when used over a reasonable period of time, needs a configuration file to define which channels to use, what their scale factors are, their names, and units, etc., etc. The data recorded in “Run 107″ was recorded using “JOEs setup”, but the data recorded in “Run 108″ was recorded with “JOEs Other Setup”. So how do you keep the files paired? You don’t. You can try various naming schemes, but sooner or later, some mistake will leave the user with a missing or mismatched CONFIG and DATA file pair.

I avoid that whole scenario by including the config data structure inside the data file. Every data file contains the config used to record it. You just put the config cluster inside a large cluster that includes your data, and record that. There is no question about which scale factor was used on the flatistrat channel, because it’s recorded right there. It’s a bit wasteful in terms of disk space, but not terribly so.

Datalog + Binary = Hybrid

So, given all that, suppose you have a LOT of data to record. The config data is a small portion, and the data is huge. There’s a problem with the idea of a data file being a cluster containing the config structure and the data. And that problem is memory size. To write a cluster to a datalog file, you have to have the data all in one place. If your data is stored in some other place as it’s acquired, then writing the file means making a COPY of your huge data and putting into the file cluster before writing the file. That’s wasteful. And back in the days when 8 Megabytes was all the RAM a machine could hold, and my clients needed to record more, it wasn’t even POSSIBLE.

To solve those issues, I invented a hybrid file. That term is my own label; it is not an official LabVIEW term. The idea is that you write the config data as a DATALOG file, and close it. Then you open the SAME file as a binary file , skip past the datalog portion, and write binary data. You get the benefit of both worlds: it’s easy to read / write the config header with ordinary datalog operations, and it’s easy to read the binary data with binary operations. You can write the data as you need to; you don’t need to make copies. You can write more data than you have RAM for. You just have to remember that the file doesn’t start at offset zero. It’s perfect, right?

Almost.

You have to figure out where the start of binary data should be, and that’s not trivial. LabVIEW’s DATALOG files include their own header, and the structure and size of that header is not public information. However, you can make some deductions. Since the FILE DIALOG can discriminate between datalog files of different structure, the structure format has to be embedded in the file itself. So what I do is flatten the datalog portion to a string and get it’s size. I take the TYPE STRING (which is not really a string) that comes out of the FLATTEN function, and flatten that and get another size. I add those two sizes together, and round it UP to the next highest multiple of 4096 or so. That works for any size structure, as it includes an estimate for the Datalog header, as well as our own header.

When you read the file, you do the same thing, and compute the offset where the binary data starts.

One more gotcha. Occasionally, National Instruments changes the format used in Datalog files. Usually it’s a minor change, and usually LabVIEW handles it automatically. You may have seen the message “This file was recorded using an older version of LabVIEW, it must be updated to be read. Do you want to update it?”. All that is well and good if it’s a plain Datalog file, but if it’s a hybrid file, LabVIEW doesn’t know anything about the binary data you’ve stuck on the end. So it will open the datalog portion, and re-write a new datalog portion, truncating everything after that, including your data. Beware.

Still this method brings more benefits to the table that it brings problems, so consider it for your own projects.

Enjoy.