Categories

Tags

Why We Upgraded to AWS

Once upon a time (way back in 2007) as Steve Jobs was launching the very first Apple iPhone, the team here at Tendril launched our own bleeding-edge technology that propelled us into the ever evolving Demand-Side Management space. Officially named the “Tendril Network Operating Platform” and affectionately referred to as TNOP, we had developed the infrastructure and tech stack that allowed us to perform cutting-edge services within the Internet of Things and smart energy arena.

Today, we’re working with some of the world’s largest energy providers and real estate search sites. Our engineering team has since migrated away from our beloved TNOP to Amazon Web Services (AWS), which has provided a multitude of benefits and allowed us to remain on the forefront of innovation in energy analytics. In fact, AWS recognized Tendril as one of their Hot Start-ups for 2017, read more about it here.

TNOP was a legend of its own time - performing real-time monitoring and dissemination of energy consumption and production data - a critical component of our platform to this day. As time progressed, however, our traditional data center model began to show its limitations because it simply could no longer do what we needed it to do, and understandably so: Our core engineering focus went from managing and learning from 10’s of thousands of homes every month, to calculating energy consumption across millions of homes in the United States, Europe and Australia multiple times a day.

In 2013, we made the decision to re-architect our platform to a Service-Oriented Architecture (SOA), then decided to make a full migration to the cloud two years later. By May of 2016, our cloud-based SOA had been fully deployed.

Shifting from self-controlled, in-house hardware to partnering with a cloud service was a no-brainer. But moving to a cloud environment wasn’t the hard part. The challenge was finding the right cloud partner that would give us the flexibility, agility, and ability to globally replicate environments that we needed. AWS was the clear winner on all fronts. Aside from their universally recognized name, AWS provides multiple zone failover, the ability to scale globally with like environments, and five nines of service reliability.

With AWS we’ve been able to take scalability to a new level, which has enabled our data science team to exponentially accelerate the development and improvement of our platform. With our previous system, our modeling was limited by the capacity of our hardware, so analyzing sets of big data that required a large amount of memory would sometimes become an issue. For instance, our solar model - used to analyze and forecast solar adoption - would take around twelve hours to train and require a large portion of our machines’ memory. When we acquired additional data sets to further fine-tune this model (such as knowing exactly where people were installing solar at certain times), we couldn’t easily incorporate them simply because our current hardware lacked the memory we needed. AWS has given us the capability to scale our memory capacity up as needed and in real-time, meaning we can now continually add new data to our models as soon as it becomes available. This enables us to maintain the caliber of excellence our customers have come to rely on from the Tendril platform. In fact, our refined solar model was featured last year in the Economist - check it out here.

Furthermore, iterations to our models that would have taken months on our traditional system, now take no more than a few hours. For example, fine-tuning our models on our own hardware needed to be done by applying one change at a time. Our data scientists would train a model, change a variable, train it again, change a new variable, train it again and so on until the model was properly calibrated. Now with AWS, our data science team can train our models using multiple versions and test numerous variables simultaneously. Instead of having to iterate over time, we can do so all at once.

I’m astounded by how far we’ve come in the last decade. I look forward to the next 10 years of innovation from our team, our platform and the architectural foundation that supports it.

 

Related

Categories

Tags

  • Continuous Demand Management
  • Customer Ops & CSAT
  • DERs
  • DSM
  • Data Analytics
  • Demand Response
  • Disruption
  • Energy Efficiency
  • HERs
  • High Bill Alerts
  • Privacy & Security
  • Real Estate
  • Smart Home
  • Solar

    Subscribe to This Feed