What is Project Haystack?
Project Haystack is an open source initiative to streamline working with data from the Internet of Things. We standardize semantic data models and web services with the goal of making it easier to unlock value from the vast quantity of data being generated by the smart devices that permeate our homes, buildings, factories, and cities. Applications include automation, control, energy, HVAC, lighting, and other environmental systems.
So what does this mean?
What is data? Today's data is a lot different from say, 30 years ago! With the adoption of increasing amounts of computerised and automated technologies, we want to record overwhelming amounts of data. We want to know how many steps we've walked in a day, or how many times we moved during sleep. We want to constantly monitor temperatures or how many people are using each room in a building. Maybe we've left our freezer door open and we want to know about it before all of the food thaws out!? If we get a burst pipe, we want to know before too much water has soaked through the floorboards in our house!
We record more data in our personal lives than ever before, becoming frustrated when we run into situations when we cannot find that one email you need from last year, or a photo that you know took last week. So, imagine how this translates to the commercial world. In fact, our data preferences are essential for targeted marketing and advertising and our personal data and contact information is extremely valuable to the retail industry.
But what is data?
It can be a number, like your age, your phone number, the number of people in your household, or how much you typically spend on a regular shopping item.
It can be text (or a string), with your name, address, gender, race, the car model you own, your job title or perhaps a medical condition.
It can be as simple as a yes/no indication (Boolean) or an on/off status. Or consider a date/time stamp for some action that has been logged.
2. Machine readable
To enable automation, our data needs to be machine readable. So, we could record and read all of this data from someone, but how would a computer understand all of this? How would it know which part is an address and which part is a job title for example?
In very simple terms, more systems now rely on data tagging for this. If the data type passed around is just a number, let's say for age, this is much easier for machines to process than a number - with text; for instance 36 compared to 36 years.
To understand that this is an age, we simply attach some tags to label the data. Perhaps we can add the label "age" and we can add the unit of "years". Or, if we look at temperatures, we can add a label of "temp" and add a unit of "degrees celsius". Maybe we also add a tag of "water heater" and maybe also "bathroom". In just a few tags, we can really start to identify what this data actually relates to - in a very neat and concise way. More importantly though, it enables machines to understand what the data relates to, as we implement more technology around machine learning and even artificial intelligence.
This kind of data tagging makes human language differences much easier to overcome too. Lots of tags can be universally adopted, like units, creating far less work to translate data manually, or at the user end where someone needs to actually read it all!
3. Project Haystack
Project Haystack is a global standardisation of using data tags, so that the same tagging can be read by ANY machine. On an international level, there are huge benefits - especially in the Smart Buildings industry where new devices are produced all around the world with a need to share data.
Many businesses already process more data than they can realistically handle, and changing all of this to adopt this Project Haystack standard is often too big a task for some to accomplish. The result is that there are many companies out there, in a kind of half state. They want to use Haystack tagging, but without an automated process to handle this, the task is very grand and also very specialised.
You can find out more on the Project Haystack website, or watch this video from Steve Eynon as he gives a presentation on the new version 4.0 at a Haystack event in 2019.