1. Platform Based Test Architecture
Start with Sammi

2. Agenda Introduction Why platforms matter?
Why a good architecture matters?
Tools and Architectures
Software tools, Object Oriented paradigm
General Architectures
More Platforms
Automated Test with TestStand
Systems Management and Data Management with SystemLink
So our agenda today will be a bit ambitious, and it is geared to arm you with the basics of both why and how to approach a platform-based solution.
Quick introduction on who DMC is

3. DMC Overview
Established in 1996, DMC serves customers worldwide from offices in Chicago, Boston, Dallas, Denver, Houston, New York, Seattle, and St. Louis
Established in 1996, offices in Chicago, Boston, & Denver and customers throughout the world
employees & growing
180+

4. Test & Measurement Certifications
NI Alliance Partner, over a dozen! CLA and CLD certified engineers across the company

5. Platform Based Test Architecture
Don’t reinvent the wheel.
Take advantage of problems that have already been solved by others.
**Switch Presenters -> Eric**
What do we mean by platform based test architecture?
If you want to check your , you don’t write your own client, you use Outlook or Gmail. If you want to sample a sine wave, you don’t write your own hardware drivers in assembly, you use NI Hardware, LabView, and DAQMx, for example. These pre-build platforms save you time and money, keeping you from solving a problem that’s already solved. You could still choose to write your own software from scratch, but you don’t need to.

6. Pitfalls! Common Pitfalls “This is the best tool we know of”
“Good enough for now”
“We’ll cross that bridge later”
“We didn’t know that tool already existed”
We’ve seen hundreds of customers and projects over the years, and seen many scenarios that have fallen into pits of various forms, because they didn’t have the right architecture, or the right platforms, or both.
“This is the best tool we know of”
“Good enough for now”
“Let’s just program this quickly to get it up and running”
“We’ll cross that bridge later”
“Oh, I didn’t know someone had already done that”
In summary, there are two simple reasons why this happens.
Not knowing about options, and not being armed to choose them
Not investing in the solutions to be built to last
So the solution to this is simple, and is our #1 take-away for this presentation:

7. #1 Take Away Understand your options. Invest in your solutions.
(Your future self,
and your future organization,
will thank you for it.)
Understand your options
Invest in your future.
It’s important to know the tools available already and to know which platform is right for your project both now and further down the road.
If your project doesn’t last more than a couple of days and never changes, then you probably do not have to pay attention.
For the rest of us… platforms are key, and it will payoff to put thought and investment into the tools used.

8. Test Software Architecture
**Switch Presenters -> Sammi**
Let’s start talking about how software architecture fits in to build scalable connected solutions.
We’ve all been in this situation – you have a piece of hardware on your desk and you have to communicate with it.

9. Measurement Abstraction
Test Software Architecture
Measurement Abstraction
Hardware Abstraction
So you write some drivers for it, and you write code to take measurements with it.

10. Test Software Architecture
Ad-Hoc Testing
Operator Interface
Measurement and Analysis Data Storage
Test Configuration
Stimulus Control
Test Step
Test Step
Measurement and Analysis
Test Step
Test Step
Stimulus Control Reuse Libraries
Meas. and Analysis Reuse
Libraries
Measurement Abstraction
Hardware Abstraction
Then you’re told that the client wants to be able to run some ad hoc tests on it. So we build in an operator interface, we add some stimulus control, maybe a PID, and some simple measurement recording. We’ve all done this, no problem. Nice job, we’re done! Oh wait, one more thing…

11. Test Software Architecture
Ad-Hoc Testing
Automated Testing
Operator Interface
Measurement and Analysis Data Storage
Pass/Fail Data Storage
Operator Interface
Test Configuration
Test Sequencer
Test Sequence Configuration
Test Step Reuse
Libraries
Stimulus Control
Test Step
Test Step
Measurement and Analysis
Test Step
Test Step
Test Steps
Test Step
Test Step
Stimulus Control Reuse Libraries
Meas. and Analysis Reuse
Libraries
Test Step Configuration
Measurement Abstraction
Measurement Abstraction
Hardware Abstraction
Hardware Abstraction
Now they want to go from manual testing to automated sequences built ahead of time to run? So we take the reusable components of the ad hoc tests and build them into blocks that can run in a sequence, and we add a way to configure these parameters, we store results automatically as they run in an organized way, we separate test results from different runs. AND we keep the ad hoc functionality.
Take a breath, you finished!

12. Test Software Architecture
Systems Management
File Movement
Dashboards
Deployment
Analytics Platform
File Cache
Measurement and Analysis Data Storage
Pass/Fail Data Storage
Operator Interface
Operator Interface
Test Sequence Configuration
Test Configuration
Test Sequencer
Test Step Reuse
Libraries
Test Step Configuration
Stimulus Control
Test Step
Test Step
Measurement and Analysis
Test Step
Test Step
Test Steps
Test Step
Test Step
Stimulus Control Reuse Libraries
Meas. and Analysis Reuse
Libraries
Measurement Abstraction
Measurement Abstraction
Hardware Abstraction
Hardware Abstraction
The client loves it! They want to put it on multiple test stands and now you have to manage multiple remote systems all at once! They would also love to be able to see data from all systems in one place, maybe they want to add user management for who can see that data.

13. Test Software Architecture
Data Management
PDF
Data Preprocessor
Data Search
Analysis Server
Enterprise Server
Reports
Systems Management
File Cache
File Movement
Dashboards
Deployment
Analytics Platform
Measurement and Analysis Data Storage
Pass/Fail Data Storage
Operator Interface
Operator Interface
Test Configuration
Test Sequencer
Test Sequence Configuration
Test Step Reuse
Libraries
Stimulus Control
Test Step
Test Step
Measurement and Analysis
Test Step
Test Step
Test Steps
Test Step
Test Step
Stimulus Control Reuse Libraries
Meas. and Analysis Reuse
Libraries
Test Step Configuration
Measurement Abstraction
Measurement Abstraction
Hardware Abstraction
Hardware Abstraction
And now they want to add reporting, and the project has grown and grown with you struggling to keep up and the program struggling to adapt even more.
I’m sure this has happened to all of us, and it illustrates the importance of taking time initially to set up a good architecture that can adapt to inevitable change.

14. What makes a good architecture?
It works
It’s scalable
It can be debugged easily
It supports a team
It survives multiple future developers
It is well documented
**Switch Presenters -> Eric**
Now we’ve talked about why a good architecture is important, let’s talk about what makes a good architecture.
The most important thing is: it actually works and satisfies the base requirements.
The solution is scalable. By that, I mean it’s flexible enough to adapt to future requirements.
It’s easy to debug, which means that it’s logically organized such that other developers can follow what’s going on.
It supports a team of developers, so that work can be parallelized and developers focus more on developing and not conflict resolution.
Once your team is done with it, it will support future developers. If the solution is too complicated, or opaque, or simply unfun to use, there are good chances that it will get scrapped or avoided, or replaced.
It’s well documented. Good does not mean just thorough, good means that it’s accessible and informative without being burdensome)

15. Test Software Architecture
PDF
Enterprise Server
Reports
Ad-Hoc Testing
Automated Testing
NI HW Drivers
NI HW Drivers
With this “map” of our big picture in mind, we can start looking at how specific National Instruments software tools and languages that can be utilized to achieve a good architecture.
On both the left and right sides of this map, we see a tool that we’re all familiar with: LabVIEW. In fact, I’m guessing that most of your first instincts would be to fill all of these boxes with LabVIEW. And that’s fine; LabVIEW is a flexible tool that is applicable to many parts of your system. My goal here today is to tell you about other software tools that NI offers as an alternative to LabVIEW that might save you time and money.
Later in this presentation we will give a broad overview of a couple of these tools, with general recommendations on usage.

16. Test Software Architecture
PDF
Enterprise Server
Reports
Ad-Hoc Testing
Automated Testing
NI HW Drivers
NI HW Drivers
With this “map” of our big picture in mind, we can start looking at how specific National Instruments software tools and languages that can be utilized to achieve a good architecture.
On both the left and right sides of this map, we see a tool that we’re all familiar with: LabVIEW. In fact, I’m guessing that most of your first instincts would be to fill all of these boxes with LabVIEW. And that’s fine; LabVIEW is a flexible tool that is applicable to many parts of your system. My goal here today is to tell you about other software tools that NI offers as an alternative to LabVIEW that might save you time and money.
Later in this presentation we will give a broad overview of a couple of these tools, with general recommendations on usage.

17. Test Software Architecture
PDF
Enterprise Server
Reports
Ad-Hoc Testing
Automated Testing
NI HW Drivers
NI HW Drivers
**Switch Presenters -> Sammi**

18. Interoperability Options
Call Library Function Node
.NET Constructor Node
LabVIEW-built Shared Lib
LabVIEW-built Shared Lib
.NET
LabVIEW 2018 Python Node
Python/LabVIEW Connector by Wineman Technologies
Python C API
IronPython
Python ctypes module
Python ctypes module
In the spirit of good architecture, organization, and reusable code, let’s talk about driver abstraction.
Through driver abstraction, you’re not tied to one language.
Software languages don’t live in isolation; you can write different parts of your application in different languages based on requirements, developer abilities, or reusable code (someone already wrote this in python etc).

19. MODULAR INSTRUMENT DRIVERS
NI Driver Support
DATA ACQUISITION AND
MODULAR INSTRUMENT DRIVERS
.NET
For example, maybe you find pre-made drivers for the hardware you’re using, but they’re not written in labview. You don’t have to rewrite them! They can likely still interface with your data acquisition application.

20. NI Driver Support
.NET
Conversely, you can use other languages to speak to NI drivers. Take for example, DAQmx. DAQmx is an awesome hardware driver that allows your LabVIEW code to easily interface with your NI hardware. But if for whatever reason you can’t use LabVIEW for your application, maybe your company prefers their testing software to be built on the .NET stack, you can still use your NI hardware because of the compatibility of DAQmx.

21. LabVIEW OO Ad-Hoc Testing Automated Testing
Operator Interface
Measurement and Analysis Data Storage
Pass/Fail Data Storage
Operator Interface
Test Configuration
Test Sequencer
Test Sequence Configuration
Test Step Reuse
Libraries
Test Step
Stimulus Control
Test Step
Test Step
Measurement and Analysis
Test Step
Test Steps
Test Step
Test Step
Stimulus Control Reuse Libraries
Meas. and Analysis Reuse
Libraries
Test Step Configuration
Measurement Abstraction
Measurement Abstraction
Hardware Abstraction
Hardware Abstraction
Here, we want to add a quick architecture note about LabVIEW OO, or Object-Oriented programming.
In the spirit of reuse, and abstraction, it can be extremely useful to use an object oriented paradigm in your approach. And we at DMC highly encourage it.
Using a programming paradigm like OO will enable you to leverage familiar design patterns that many developers will recognize. You can then choose an architecture that promotes scalability and flexibility…
in other words, the software will be more capable of tolerating change in both features and development team.
Going back to our big picture architecture, a classic application for Object Oriented is in Measurement and Hardware Abstraction (HAL/MAL).
Let’s say you have a power supply, and you write your program with VIs that are capable of communicating with that specific power supply. Everything looks great! But wait, your client has added a different type of power supply! Do you have to rewrite your application?
If you had started with an object oriented approach, you would have made a generic “power supply” class that your application uses. It has a child class that contains methods specific to a certain power supply. Now, all you have to do is add a new child class for the new power supply. The application itself remains unchanged!
I could spend an entire session about how to use an object oriented approach in LabVIEW, but luckily for you guys several other people today are doing just that and I encourage you to attend those sessions if you would like to learn more.

22. LabVIEW OO LVOO is a simple way to improve: Code reuse
Test Step
Test Sequencer
Test Steps
Measurement Abstraction
Operator Interface
Test Step Reuse
Libraries
Hardware Abstraction
LVOO is a simple way to improve:
Code reuse
Software flexibility
Software scalability
Maintainability
Collaboration
OO is a powerful tool, independent of your architectural decisions!
Defining and separating your devices using OO is highly recommended, and can be a building block for teams of developers, extensive reuse, and future modification.
Addresses issue of LabVIEW classically not being very multi-developer friendly: with multiple classes, each developer can work on a different class without conflicts.

23. Test Software Architecture
PDF
Enterprise Server
Reports
Ad-Hoc Testing
Automated Testing
NI HW Drivers
NI HW Drivers
**Switch Presenters -> Eric**
Talk about Flexlogger briefly here.
For ad hoc testing, we’re just doing basic IO so we can use LabVIEW for that, or we can use NI’s FlexLogger. FlexLogger is a simple data acquisition and logging tools

24. Test Software Architecture
PDF
Enterprise Server
Reports
Ad-Hoc Testing
Automated Testing
NI HW Drivers
NI HW Drivers

25. Test Software Architecture
PDF
Enterprise Server
Reports
Ad-Hoc Testing
Automated Testing
NI HW Drivers
NI HW Drivers

26. TestStand Create, execute, and debug test sequences
Reuse code developed in other programming languages
Generate reports and integrate with databases
Develop or connect to professional operator interfaces
Think back to the beginning of this presentation when I said you could write your own client, but you don’t have to. The same is true for the test and measurement space; you could write your own sequencer, but you don’t have to because NI has already done that.
TestStand is a basic sequencing application that allows you as the developer to focus more on defining sequences rather than developing a test executive and a sequencing environment. Here are some of the out-of-the-box features that TestStand offers:
You can develop code in other programming languages (think hardware drivers, accessory applications, etc) and have them called from a TestStand sequence.
TestStand has report generation and database integration as well. The TestStand engine collects data like statuses, error, and pass/fail results while a test executes and will generate prebuilt reports in a variety of text-based formats. If you want to store this data in a SQL databse, TestStand can do that too.
You can opt to run tests directly from the TestStand sequencer editor, but you can also use LabVIEW to create better, more attractive and easier to use interfaces. Maybe you can take some of the tips that Steven Dusing shared in his presentation on good UI practices in LabVIEW.
TestStand is a basic sequencer that’s useful if you have many projects with similar, simple testing logic. Employing TestStand in this scenario can save you from needing to create the same thing over and over again, especially if you don’t need functionality outside of TestStand. I know that there are probably some strong opinions on this software, but our goal here isn’t to tell you what’s best for your application, but rather what’s available.

27. TestStand TestStand is a good choice for:
Running rigidly-defined tests (e.g., validation, EOL tests)
TestStand is not the best choice for:
Tests involving deterministic Real-Time systems
So now you know some of the main features of TestStand, but when should you actually use it?
Test stand is good for running basic automatic tests that don’t need to be dynamic during runtime
However, you’ll want to look for alternatives if you’re running tests on deterministic real time systems or tests that require specific custom implementations. Many organizations who need a more flexible solution have opted to create their own test sequencing software. At DMC, we’ve created a sequencer written in LabVIEW that has become our internal standard for sequencing projects which addresses some of TestStand’s shortcomings like its inability to run on real time targets. Fresh Consulting has their own sequencer called the ScreenPlay framework, which you can learn more about at their presentation later today. And as we talked about before, LabVIEW supports interoperability. This means if your organization has created a generic sequencer in .NET or Python, you can still use it to automate your LabVIEW testing application.

28. Test Software Architecture
PDF
Enterprise Server
Reports
Ad-Hoc Testing
Automated Testing
NI HW Drivers
NI HW Drivers

29. Test Software Architecture
PDF
Enterprise Server
Reports
Ad-Hoc Testing
Automated Testing
NI HW Drivers
NI HW Drivers
**Switch Presenters -> Sammi**
Going back to our architecture overview…
Let’s now look higher up on this hierarchy and talk about tools for data and system management.
Specifically, let’s talk about National Instruments SystemLink.

30. The Challenge of Managing Distributed Systems and Data
Calibration
Maintenance
Utilization
Deployment
Test Configuration
Configuration
Test Monitoring
Data Transfer
Data Visualization
Data Reporting
Notifications
Alarms
Health Reporting
Performance
Test Analytics
CONNECT AND DEPLOY
MANAGE AND ANALYZE
Anyone who has spent time programming software for Test & Measurement applications knows that managing these applications can become challenging quickly.
First you develop the custom application– that’s fun, right? That’s the part we all are good at doing.
Now you need to deploy to multiple targets and keep them all up to date. For some projects, this could mean spending half a day walking around to each station with a flash drive. Which is time that could be much better spent on the actual debugging rather than on the release.
And you have to keep track of which version is on which machine
And now you also have to keep track of which versions of other software is on those systems… like NI VISA, or 3rd party tools.
Or, adding in diagnostics to monitor the health of your test equipment. If an enclosure fan gets clogged and stops being effective, how will you know before it’s too late and your Real-Time PXI controller overheats and reboots? So you take more time and program this into your custom application.
If you’re logging data, do you have to go around and pull files off of each station one by one?
These tasks start taking a lot of time. And are less fun.
And that’s valuable time that you would *much* rather spend operating in your domain expertise– actually working on the core code that is of the largest benefit to testing your product.
Historically, this has been a bit of a gap– test engineers and organizations have not had great tools for these higher level needs.

31. Features at a glance Build systems faster. Manage them better.
SystemLink allows you to manage and monitor:
System health and performance, with alarms and notifications
Software deployments and software configuration management
Automated test procedures with sequence monitoring and user-defined dashboards
Data transmission and visualization with open APIs and graphical interfaces
Data analysis and report scheduling with search, standardization, and 3rd-party plug-ins
Product Capabilities
Manage a group of networked devices
Access the application via web browser
Support NI and non-NI SW and HW
Leverage extensible, plug-in architecture
So this is SystemLink which is mean to make managing distributed systems easy.
This software enables engineers to connect, manage, and deploy software to remote fleets of devices. It’s not targeted for a specific industry, so it is widely applicable.
It shows the version that each device is currently running as well as the current performance.
SystemLink enables management of your test and measurement systems from any location because it’s all through a web browser.
It allows for easy setup of dashboards for data visualization.
Mention Steven and Beth’s presentation on SystemLink at NI week this year?

32. Distributed Tilt Table Demos
Tilt table demos for career fairs, so they get spread all over the country. To easily push updates to all, we keep them on a SystemLink server.

33. Distributed Tilt Table Demos
Upload packages to feeds (packages contain the exe and supporting software). A cRIO for example could subscribe to a feed and updates could be pushed to it in this way.

34. Distributed Tilt Table Demos
Lists all software currently deployed on a given device.

35. Distributed Tilt Table Demos
Lists the software version with the ability to deploy other versions (previous? Good source control?)

36. Distributed Tilt Table Demos
Can read specific tags from each device, in this case location. Question: is this done in LV? Is this part of the device configuration when you add to systemlink?

37. Platform Based Test Architecture
Questions? Contact Us!
The purpose of this presentation was to talk about how you can leverage existing software and platforms to save yourself from solving an already solved solution. By using verified and extensible platforms, you can focus more on your unique domain expertise, resulting in more flexible long-term solutions for your applications.
That concludes our presentation today on Standardizing on a Platform-based Test Architecture.
Thanks for joining us, and we invite you to reach out to us to learn more. We also will have a team at NI Week 2020, and will have a few presentations at the event in case you are attending. Also, if you’re from the area, we encourage you to attend the quarterly seattle labview user group at Fresh. It’s always a great place to meet other local developers who have probably already solved something you’re looking to do and learn something new about the labview.