Seenit Classic Logo

Seenit Enterprise (Classic) is our industry leading UGV Platform, paired with our iOS and Android Capture apps.

Go to Support Centre
Seenit Stellar Logo

Stellar is our new web-based platform. If you are trying to upload to a story, please follow the link you were sent.

Go to Help Centre

Not sure which product you are using? Find out more

Seenit Classic Logo

Seenit Enterprise (Classic) is our industry leading UGV Platform, paired with our iOS and Android Capture apps.

Login to Seenit Classic
Seenit Stellar Logo

Stellar is our new web-based platform. If you are trying to upload to a story, please follow the link you were sent.

Login to Seenit Stellar

Not sure which product you are using? Find out more

Seenit Classic Logo

If you were sent a Project ID then you can contribute by downloading the Seenit Capture App below.

Seenit Stellar Logo

Stellar is our new web-based platform. If you are trying to upload to a story, please follow the link you were sent.

Troubleshooting

Update from the Seenit tech team - how we keep our customers' video dreams alive

Ian Merrington

18 November, 2018 • 

Seenit Tech Report Q3 2018

Seenit Reliability

At Seenit we take the reliability of our platform very seriously. We use the full range of tools offered by the Google Cloud Platform and Couchbase to allow us to develop and operate our platform with no disruption to our customers and collaborators.

The philosophy we have used is simple, keep the complexity down, leverage Google Cloud features where possible, rely on multiple levels of redundancy and automate everything. This is all monitored by the best in class monitoring provided by Datadog. All of this means we can spend our time focusing on improvements rather than fighting fires.

In order to achieve our desired platform availability Seenit utilises the following;

Multi Data centre Deployment

By using multiple Data centres we can protect ourselves from issues affecting one datacentre.

Autoscaling Node pools

Each external facing application is run as a Autoscaling Node pool. This means that we have a pool of web servers, all accessed via a Google Load Balancers that expand and contract with the load on the platform. Any issues with any web server will be detected by the load balancer and removed, so as to not impact the the users.

Autoscaling Kubernetes for the backend processing.

We love tools that make our life easier, and Kubernetes has improved the deployment and scaling of our backend video processors. 1 Click rollout of new releases, autoscaling and auto restart on failure gives as a very resilient platform. Issues with any server and the load is moved to another server automatically. In order to prove our platform is resilient we use a Chaos Monkey approach.

Distributed Database

With any platform, your data is key, and because of this we utilise a Distributed multi data centre database, Couchbase, and a clustered Message Broker (RabbitMQ) for intersystem communication. Couchbase provides scaleability and resiliency due to its distributed architecture. Its has cross data centre replication, so you always have a current copy of the data in your offline data centre. Couchbase will detect any issues with a server and remove it from the cluster with minimum distribution to the service.Its also very simple to deploy and manage, giving us more time to focus on other things.

Monitoring

All systems are monitored by DataDog Monitoring for rapid visibility of any potential issues. Our engineers can be notified of any issues via Slack or email before they become service impacting.

Testing and Deployment

It’s not good having solid infrastructure if the code isn’t up to standard or if you cause issues deploying new releases. All code has to go through our rigorous test pipeline, and once the code has passed, our automatic deployment tools will roll them out to production, and more importantly, if there are any issues, the same tools can restore the old release, to minimise customer impact.

Our Q3 review returned the following results

Uptime 100% - No outages

Successful Studio Requests 99.994% or 30 errors per month out of 800 000 requests

Uploads 99.3% Processed Successfully

We are constantly looking to improve this, and having just finished the Kubernetes migration, we are now looking to improve our automatic reprocessing of failed uploads, as we know how valuable your videos are.

Other posts you might like

Latest in News

Trending on Seenit