← Back to site

GitOps For The Distributed Enterprise Edge

Darryl Cauldwell

What is edge computing?

Edge computing concept

Royal Mail IT Services 1993 – 2003

Royal Mail
Distributed Edge

37 Sorting Offices & 1,356 Delivery Offices

  • Windows PCs running track & trace, route planning, revenue protection and MS Office
  • “Man in a van” systems administration
  • Poor configuration management, slow rollout, very expensive
Distributed Far Edge

48,000 Vehicles — 1,110 Parcelforce

  • Analogue carphone, modem and compute device
  • Send back to central service centre for application lifecycle
  • Lifecycle management was slow and expensive

VMware — 2019 to present

VMware

Customer Success Architect

  • Cloud Foundation: vSphere, vSAN, NSX, Aria suite
  • Tanzu: App Modernisation, Spring, Cloud Foundry, Kubernetes
  • Telco Cloud: RAN Disaggregation, Private 4/5G, OpenStack
  • Secure Access Service Edge: SD-WAN, Edge Intelligence
Office of the CTO

Office of CTO Ambassador

  • Sustainable Software
  • Serverless (Knative)
  • Edge

Near Edge

Few nodes with shared storage, cluster for availability and resilience. Data centre WAN connectivity and management plane. Optimised for low latency applications.

🏭

Warehouses

  • Quality Inspection — Computer Vision
  • Theft prevention — Computer Vision
  • Logistics
⚙️

Factories

  • SCADA & Intrusion Detection
  • Predictive Maintenance — Computer Vision
  • Quality Inspection — Computer Vision
  • PLC & Physical security
⚡

Electricity Grid
Transmission

  • Protection automation control
  • Overhead line telemetry
  • Intelligent Electronic Devices
  • Physical security — Computer Vision

Distributed Edge

Single or small number of nodes with resource constrained hardware. 4/5G connectivity, SD-WAN. Optimised for low latency applications.

🛒

Retail

  • Point of Sale — Computer Vision
  • Theft prevention — Computer Vision
  • Quality Inspection — Computer Vision
  • Inventory management
🚢

Vehicles / Ships

  • Predictive Maintenance — Computer Vision
  • Physical security — Computer Vision
  • Signals & Telemetry
  • Drone operation
🔌

Electricity Grid
Distribution

  • Protection automation control
  • Intelligent Electronic Devices
  • Physical security — Computer Vision
  • Worker Safety — Computer Vision

Wind Turbines Predictive Maintenance

  • 72,000+ turbines
    • Management at scale
    • Observability at scale
    • Low cost commodity hardware
  • Consistent infrastructure deployment and configuration
  • Rapid, consistent iteration and delivery of computer vision inference models
  • Automated, zero touch provisioning
  • Secure 5G connectivity
Wind Turbine Predictive Maintenance Architecture

Why data processing at the edge?

  • Real-time decision making
  • Increase app reliability, responsiveness and security
  • Reduce bandwidth and infrastructure costs
  • Enhanced customer experience
  • Enhanced data privacy and security
“55% of all data analysis by deep neural networks will occur at the point of capture in an edge system by 2025”
Source: Gartner Identifies Top Trends Shaping the Future of Data Science and Machine Learning, August 2023

Common edge constraints

  • Application — Many existing apps may not be modernised for years. Configuration drift. Lifecycle management complex.
  • Limited compute — Inelastic resources. Apps tied to specific hardware. GPUs tied to specific applications.
  • Organic growth — Disparate hardware and operating systems.
  • Limited network — High latency, low bandwidth. Potential outages to central management.
  • Siloed data — Pockets of data distributed in silos.
  • People skillset — Few trained in IT at edge locations. Few dedicated to Edge in central IT.
Constraints

Image: jooinn.com

VMware Edge Cloud Orchestrator

  • Zero touch provisioning of infrastructure and applications
  • GitOps workflows for day zero and day two operations
  • Infrastructure and applications desired state manifest files in a git repository
  • VM and container based applications run side-by-side
  • Consumer grade hardware

Tech Showcase — xLabs

xLabs is a program within the Advanced Technology Group in VMware’s Office of the Chief Technology Officer. Cultivating cutting-edge technologies in collaboration with partners and customers.

VECO Architecture

VECO Architecture - Management Plane connected to Control and Data Planes

Live Demo

VECO Console

veco.showcase.vmware.com

GitOps Workflow

Git

Declarative source of truth for entire stack

firmware.yml • esxconfig.yml • vm.yml • container.yml

→
Flux

Update to source of truth triggers pipeline

→
Edge devices with VMs, Containers and ESXi Config

Configuration pulled and realised by each device

Live Demo

Git Repository

github.com/darrylcauldwell/keswick

Zero Touch Provision and Operate

1
Ship — OEM ships edge box(es) to site
2
Plug in — Person at site connects edge box(es)
3
Setup — Edge agent contacts cloud service, authenticates, pulls configuration, updates itself
4
Day 2 — Edge agent notices mismatch with desired state and acts to resolve

ESXi Operator in Kubernetes

kubectl get crd | grep esx
hostconfigurations.esx.vmware.com

kubectl -n esx-system get HostConfiguration
NAME
esx-base-profile
keswick-host-config

VMware wrote a Kubernetes HostConfiguration Operator to manage ESXi host configuration. It connects to the ESXi APIs to enact configuration pulled from the git repository.

VM Operators in Kubernetes

Introduced vSphere 7.0 U2a — VirtualMachine, VirtualMachineImage, VirtualMachineClasses and VirtualMachineServices

kubectl get crd | grep vmoperator
virtualmachineclasses.vmoperator.vmware.com
virtualmachineimages.vmoperator.vmware.com
virtualmachines.vmoperator.vmware.com
virtualmachineservices.vmoperator.vmware.com
...

kubectl get vm -o wide
NAME      POWER-STATE   CLASS              IMAGE
edge-vm                 guaranteed-small   photon-hw11-4.0.ova

Live Demo

Terminal SSH — Host and VM Operators

Demo

Demo Architecture - RTSP Camera to Inference Engine stack to Git
OpenCV

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library.

OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.

Caffe

Caffe is a deep learning framework. Caffe Face Detector is an OpenCV Pre-trained Model.

Thank you

Appendix

What is OpenVINO?

OpenVINO

Appendix

OpenVINO

Open

Visual

Inference

Neural networks

Optimisation

OpenVINO architecture - frameworks to hardware

Appendix

AI Accelerators for Deep Learning Inference

  • An AI accelerator is a dedicated processor designed to accelerate machine learning computations
  • Deep learning is primarily composed of linear algebra computations that can be easily parallelised
  • Inference is often the most time consuming part of your application that directly affects user experience

iGPU

Integrated graphics processing on Intel Core and AMD Ryzen processors

dGPU

Discrete graphics processing units typically supplied by AMD and Nvidia

Appendix

OpenVINO Toolkit

OpenVINO toolkit workflow

Appendix

GPU Access Models

GPU Passthrough - dedicated GPU per VM

Passthrough

Shared vGPU - multiple VMs share one GPU

Shared vGPU

( NVIDIA NVAIE license )