# Release Orchestration ### for Fun and Profit A brief intro to Vamp's current capabilities. Note: Here is speaker notes
## The story - A basic web-service is built, tested, and deployed to users. - The service is one of many dozens or hundreds of microservices working in cooperation (think netflix) - A single team owns only a handful of services. (Modern teams using cloud native architecture on k8s platform) ![netflix UI mocked up wiot potential service boundaries](assets/netflix.png) -- ## Traditional CCI Story - Developer commits code - Fix breaking change in unit - fast feedback, optmizations - Concurrent packaging of docker while IT tests run - Deploy versioned image to docker hub - Apply deployment to K8s cluster. ![circleci workflow with build, package, apply workflow](assets/cci_workflow.png) -- ### But ah Snap! The service selector in the manifest was bogus, and our beautiful little container is shutoff from the world. Users are getting angry error pages! ![githubs 503 unicorn](assets/gh_angry.png) -- ### Adding Release Monitoring - Release agent detects deployment by CCI, Argo, or sun gods (by matching labels [app, version]) - Release agent monitors metrics from prometheus based on defined metric queries - At end of policy time-limit determine if thresholds were met and moves forward, or fails deployment - Webhook is fired with details - Admins get paged and fix stuff. ![alerts and metrics image](assets/alerts.png) -- ### Adding Traffic Shaping - Release agent also applies new ingress policy controlling traffic (10% canacy, 90% stable) - Vamp agent monitors metrics from prometheus based on defined metric queries - At end of policy Vamp determines if thresholds were met and increases rollout, or fails the `release` - Webhook is fired with details ![alerts and metrics image](assets/canary.png) -- ### So What? - If the application restarts too much OR - Does not respond in time OR - Throws too many errors while it does OR - X OR X OR x Then reset ingress to "Stable" version, and fire webhooks - operators don't get pages - only a few customers ever notice - dev team can keep iterating -- ## Webhooks, WHY DO WE NEED EM? - Vamp handles ingress & scale based on policies that shape ingress, - but leaves the deployment itself out there (unrouted) - Webhooks on Success to: - approve CCI apporval job - trigger parametized pipeline - On failure: - call CCI to cleanup bad deployment. -- ## Recap - Traditional approach was "hope our non-prod testing was adequate" - A _passing_ pipeline in CircleCI is not the same as a _healthy_ application in the wild - Invalid config, inadequate scale, integration issues, etc. - Modern approach "Ensure our non-prod testing was adequate" - We separate `deploy/publish` from `release` - We can incrementally expose or rollback new versions -- ![tired, wired, inspired meme with CI,CD and RO](assets/releasethinking.jpeg)
# Vamp Deep Dive Walkthrough the Vamp agent, applications, policies, etc -- ## Vamp Release agent Firstly, there is not direct trigger or API call to vamp. With the Vamp release agent we essentially hand ingress control to Vamp. The cluster installed binary communicates with Vamp Cloud to shape traffic based on detected deployments. ![visual of vamp agent on network](assets/vampoverview.png) -- ## Vamp - Application An *Application* represents the tuple of (Cluster, Namespace, Ingress) You may think of Application as more of a Platform - it is not a specific web-service, UI, or function, but rather everything available behind a single domain. [BlueSKyGreenBuild App in Vamp](https://vamp.cloud/6/application/211/0) -- ## Vamp - Services A *Service* reprsents an individual web service or application running as a deployment, responsing on a specific path, and many may live in an Application https://vamp.cloud/6/application/211/0/services -- ## Vamp - Policies & Metrics Metrics are queries against prometheus essnetially. Policies are which metrics (beyond required health & availability) are used over a fixed time or request volume to assess validity of change. - 90% of responses < 500ms - 1,000 requests/min of traffic - No more than 2 restarts https://vamp.cloud/6/policy/66/steps
# Cool Let's Play https://app.circleci.com/pipelines/github/eddiewebb/demo-blueskygreenbuilds?branch=main
# Fin