writeup on the package

[christopherDT]

Hi @bbartling,

As mentioned in another discussion post, I've been testing out the project, and it's really great. I've recently been writing up some data science tutorial posts on Medium that center on open-sourcing building data science. I wanted to spotlight open-fdd and help get the word out, so I've been fashioning my experiences with it into a post. Basically, it's mostly the front end of the data bottleneck: how to choose the right equipment/sensors, and how to get the data into the right format to feed to open-fdd for testing. Any thoughts or suggestions?

[bbartling]

That sounds about right...another thing this industry from the get go is very little open source with the exception of BACnet protocol stacks. And as @acedrew would say "physics.. aren't to be patented" or something along those lines. @acedrew any thoughts??!

…

On Tue, Feb 28, 2023, 3:21 PM Christopher Dudas-Thomas < ***@***.***> wrote: Hi @bbartling <https://github.com/bbartling>, As mentioned in another discussion post, I've been testing out the project, and it's really great. I've recently been writing up some data science tutorial posts on Medium that center on open-sourcing building data science. I wanted to spotlight open-fdd and help get the word out, so I've been fashioning my experiences with it into a post. Basically, it's mostly the front end of the data bottleneck: how to choose the right equipment/sensors, and how to get the data into the right format to feed to open-fdd for testing. Any thoughts or suggestions? — Reply to this email directly, view it on GitHub <#7>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHC4BHMEO6WZNOE6JNDQNVDWZZT7PANCNFSM6AAAAAAVLHXIZQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[bbartling]

@christopherDT when the Medium article gets written be sure to send it this way! My background is more expertise on HVAC systems, HVAC controls PLC programming, building automation setup/implementation, existing building commissioning, and any analyst type work in the smart building arena or existing building commissioning sometimes referred to as RCx or retro commissioning...which is a different skillset than the data engineer, data scientist, IT, and even the developers that create the BACnet stacks or full stack guys that create the smart building platforms in the cloud/IoT.

One other thing by design of this repo that I think would be cool to mention in an article online, like the Medium article you are going to create:

• The code could be used on the edge where in the smart building IoT industry, its some Linux based machine with BACnet (and other operational technology (OT) protocols) drivers or stacks collecting data from the LAN inside the building that all OT technology resides on, or Silos as mentioned in Nexus labs. The HVAC which is a major energy or fossil fuel user inside the building is just one of the "silo's" under BAS or building automation system. This is the other repository of mine of attempting to get this running on the edge in IoT in a free open source platform called VOLTTRON. Nexus labs does a really good job at attempting to steer the people in the smart building area in the right direction where hopefully someday every building will have its own API and meta-data strategy...Nexus Labs is the good Kool-aid where the building owner manages their own data and has free access to it. They also do really good stuff and attempting to rate or score card a building OT, as some buildings the OT is way too outdated and/or the HVAC/BAS is in too rough shape to even consider IoT, FDD, or making a building grid interactive or referred to as a GEB building which is a building of the future for electrification of the grid..

OR use case 2 of massive historical datasets which is open to debate:

I think this repo could be used to turn the HVAC system fault detection rule based equations defined by ASHRAE written as functions into a machine learning classification problem. For example each one of the fault rules or .py files its the same format using Pandas apply:

# return a whole new dataframe with fault flag as new col
df2 = _fc8.apply(df)
print(df2.head())
print(df2.describe())

Which then will return a few dataframe df2 with the fault flag as an int of 0 or 1. Like this where fc8_flag is all zero's in printing the data frame head():

                     AHU2_CW_ValveAO  AHU2_MA_RA_DamperAO  AHU2_MATemp  ...  sat_fan_mat  sat_mat_sqrted  fc8_flag
Date                                                                    ...
2023-01-01 00:00:00             10.0                100.0         57.9  ...          0.3        5.385165         0
2023-01-01 00:15:00             10.0                100.0         57.9  ...          0.3        5.385165         0
2023-01-01 00:30:00             10.0                100.0         57.9  ...          0.3        5.385165         0
2023-01-01 00:45:00             10.0                100.0         58.0  ...          0.3        5.385165         0
2023-01-01 01:00:00             10.0                100.0         58.0  ...          0.3        5.385165         0

So I think in theory if someone found a massive wonderful dataset of an air handling unit (AHU) with lots of faults or problems! (which could be mechanical falure, controls programming, sensor error, or poor control setpoints) this repo code could be used across a massive datasets (with the power of Pandas) where then someone I think could turn this into a machine learning problem for each fault equation. Which would be a whole new cool open source thing. I do think some proprietary platforms are doing this already where to my knowledge these are Phd mechanical engineers or architectural engineering students turning their Phd research work into a business. In the PDF folder see FDD-White-Paper.pdf this is a classic example of probably one of the most well developed FDD platforms out there where the creator developed all FDD in MATLAB where to my knowledge its one machine learning model that can handle any building or type of HVAC system, which is impressive because its a real mess in reality each building HVAC is very unique as with each BAS technician that sets up the BAS creating unique name conventions....its a data engineering nightmare I think. That PDF I think is also really well written that explains the difference between BAS alarms, FD, and FDD where there reporting features of this repo attempts to do more of the analytics or second D in FDD.

So if Kaggle created a couple competitions in collaboration for ASHRAE for time series data on building fuel use machine learning competition why couldn't they do the same thing for fault detection?

I am curious to know if a machine learning model would be better and easier like way less code with this rule based approach. This isn't my expertise but feeding data through a machine learning model can be done in Python with just a few lines of code and have that run on the edge....and if the competition was created, the data existed, and the research existed what a cool open source project that could be!

Hopefully this all makes sense feel free to reach out! Feel free to copy paste any of these words into a Medium article or ask for more : )

Ben

[christopherDT]

Awesome, this is really great information. That machine learning approach definitely sounds interesting, and pretty plausible! I added a bit into the article about your background, as I think that's helpful for context.

I'll definitely send the article along once it's published, and direct people to the github page or your contact info where appropriate.

It sounds like some of that work you're doing on VOLTTRON is similar to the work our company, Onboard Data, does -- we send client's building data from their building controllers directly to our servers, standardize/tag/normalize it, then provide it back to customers in an easily accessible API and web portal. That's one thing that interested me about this project, as the first use-case you're mentioning is pretty directly applicable to the work our company does.

[bbartling]

Cool stuff...I forgot to clarify with turning the FDD pandas data frame into a machine learning problem....I think the cool thing would be for the data scientist creating the models would be to save a model to file where then the saved model could be used in production in an IoT application. I would be curious to see if that is cool or not, at least from my experience mostly dabbling with this stuff in Python to its pretty easy to load a saved machine learning model and predict/run inference with just a few lines of code - like to me its seems very simple if the model can proved its a good one. So why couldn't a repo be expanded to include a bunch of saved ML models that are open source like Tensorflow Keras, or Sci-kit learn, Ada-boost, etc... what ever is working best?? I'm curious if this would neat idea to gravitate towards where I personally would love to dabble with this but I only know enough about ML to be dangerous : )

[christopherDT]

That totally makes sense. I'm wondering what benefit you think the ML model might have over the rules-based implementation here? Other than just having a model implicitly learn the rules we explicitly know from these equations, I'm imagining a point of interest could be a model that could better diagnose what the cause might be based on those sensor readings. The way I'm imagining it, then, the real label of interest would not be the fault flag, but instead an "issue" or "cause" flag, where faults could be tagged as "sensor issue", "mechanical failure", etc. Then the interesting thing would be if the model could classify new instances of those issues/causes based on nuances in the timeseries data that would be very difficult for people to pick out. Thoughts?

[bbartling]

@christopherDT some day in the future to experiment with ML if there is good data and well planned use cases this is what we would want to do right? https://towardsdatascience.com/machine-learning-multiclass-classification-with-imbalanced-data-set-29f6a177c1a

• multiclass classification with imbalanced datasets

For example I think it will be hard to find good data for some faults (my rough guess intuition) so some datasets may be biased to faults that are easily found in the real world. And we would want a multiclass design for labels like you mentioned one label could be "sensor error", "mechanical issues", "setpoint issues", etc.

I've dabbled a bit with Keras just for fun on the side so this is something I could hit the ground running in NN MLP.

I also uploaded the lbnl public datasets to the ahu_data and running the fault equations to see if there is anything flagged....am also looking for more datasets out there people could donate anonymously.

[bbartling]

Yes and yes keep expanding on what your talking about it's about what you mention about the model classifying rute cause, also see descriptions of the fault rules in the PDF it has some good incite. Another PDF in their also describes the difference between FD, FDD, and BAS alarm's where I think a ML model could be a bit more helpful root cause analysis where one could feed in alot more data than rule based. I would love to experiment with this... Would you know anything about open source free data sets for HVAC for FDD purposes? Like popular data sets for ML tutorials, ive heard through word of mouth public datasets exits for HVAC FDD purposes.

…

On Fri, Mar 3, 2023, 1:53 PM Christopher Dudas-Thomas < ***@***.***> wrote: That totally makes sense. I'm wondering what benefit you think the ML model might have over the rules-based implementation here? Other than just having a model implicitly learn the rules we explicitly know from these equations, I'm imagining a point of interest could be a model that could better diagnose what the *cause* might be based on those sensor readings. The way I'm imagining it, then, the real label of interest would not be the fault flag, but instead an "issue" or "cause" flag, where faults could be tagged as "sensor issue", "mechanical failure", etc. Then the interesting thing would be if the model could classify new instances of those issues/causes based on nuances in the timeseries data that would be very difficult for people to pick out. Thoughts? — Reply to this email directly, view it on GitHub <#7 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHC4BHITD3ZT7WHB72JMQNLW2JDZ3ANCNFSM6AAAAAAVLHXIZQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[bbartling]

This is what I am talking about for public datasets to test FDD algorithms against, I'll work on some unit test scripts to benchmark against this: https://www.osti.gov/dataexplorer/biblio/dataset/1881324 Hopefully it's something useful...I think it would be cool to benchmark against rule based FDD, hopefully the results are good, and then try some ML experimentation black box magic : )

…

On Fri, Mar 3, 2023, 2:56 PM Ben Bartling ***@***.***> wrote: Yes and yes keep expanding on what your talking about it's about what you mention about the model classifying rute cause, also see descriptions of the fault rules in the PDF it has some good incite. Another PDF in their also describes the difference between FD, FDD, and BAS alarm's where I think a ML model could be a bit more helpful root cause analysis where one could feed in alot more data than rule based. I would love to experiment with this... Would you know anything about open source free data sets for HVAC for FDD purposes? Like popular data sets for ML tutorials, ive heard through word of mouth public datasets exits for HVAC FDD purposes. On Fri, Mar 3, 2023, 1:53 PM Christopher Dudas-Thomas < ***@***.***> wrote: > That totally makes sense. I'm wondering what benefit you think the ML > model might have over the rules-based implementation here? Other than just > having a model implicitly learn the rules we explicitly know from these > equations, I'm imagining a point of interest could be a model that could > better diagnose what the *cause* might be based on those sensor > readings. The way I'm imagining it, then, the real label of interest would > not be the fault flag, but instead an "issue" or "cause" flag, where faults > could be tagged as "sensor issue", "mechanical failure", etc. Then the > interesting thing would be if the model could classify new instances of > those issues/causes based on nuances in the timeseries data that would be > very difficult for people to pick out. Thoughts? > > — > Reply to this email directly, view it on GitHub > <#7 (reply in thread)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AHC4BHITD3ZT7WHB72JMQNLW2JDZ3ANCNFSM6AAAAAAVLHXIZQ> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

[bbartling]

@christopherDT well this was interesting....it appears there already was a Kaggle competition for FDD haha in 2020:
https://www.kaggle.com/datasets/claytonmiller/lbnl-automated-fault-detection-for-buildings-data

Ill have to dig through this IPython notebook someone created for it.

[christopherDT]

This is great, thanks for sharing the resource! This seems like it could be very helpful in determining what next steps might look like if something were to turn into a project in this realm

[bbartling]

I also want to see how this could be used as well with testing...supposedly this project is getting a makeover. It would be really neat if fault rules could be ran across large datasets all based on meta tags Vs having to manually enter in point names, etc..
https://beta.mortardata.org/intro.html