2 min readfrom Machine Learning

Rare event prediction on time series that change structure mid-stream? [D]

Hi reddit! I made this post on r/MLQuestions, but I am posting it here too for spread:)

This is a case I have been assigned at work and I'd love input from anyone who's tackled something similar.

I'm building a failure prediction model for ~33k chargers. The devices emit data at two very different rates depending on operational state: roughly 1 obs/hour when idle and 1 obs/20s when active with a different feature set in each mode. I want to try predicting failures within a 7 day horizon, but I am open for other suggestions.

The positive rate is around 1% at 30 days and 2% at 90 days with a max of 5% of devices ever failing. Strong per-device behavioral variance makes it hard to even define what "normal" looks like. Devices have different usage patterns and

I'm now thinking about whether the mode shift problem is better solved at the architecture level or the data level. One option I'm considering is two separate RNN encoders for each operational state feeding into a shared decoder. But I'm also open to windowing and sampling approaches. And beyond reweighting and loss skewing what has actually worked for you at sub-2% positive rates in time series?

How would you tackle an issue like this?

submitted by /u/Beginning_Chain5583
[link] [comments]

Want to read more?

Check out the full article on the original site

View original article

Tagged with

#generative AI for data analysis
#Excel alternatives for data analysis
#natural language processing for spreadsheets
#real-time data collaboration
#big data management in spreadsheets
#enterprise-level spreadsheet solutions
#conversational data analysis
#financial modeling with spreadsheets
#intelligent data visualization
#data visualization tools
#enterprise data management
#big data performance
#data analysis tools
#data cleaning solutions
#rows.com
#real-time collaboration
#failure prediction model
#rare event prediction
#time series
#predicting failures