AI that works where the work happens.
Not every AI system belongs behind a cloud API. Some need to process video as it arrives, work with sensitive information, or run inside a customer’s own infrastructure. We design AI systems around those requirements, choosing where models run, what data moves, and how the application works with the results.
Where a model runs is a design decision.
There are four common answers, and one system can use several of them.
On the device
The model runs on an endpoint or a dedicated device, such as a worker’s laptop.
At the edge
Processing happens close to the cameras, sensors or other sources producing the data.
On-premises
Inference runs on servers inside the customer’s own infrastructure and network.
Hybrid
Some work happens locally and some in cloud services, split according to what each part needs.
These are architecture choices, not products. Which one fits depends on the problem, and cloud processing is often the right answer.
Engineering for the environment.
Running AI on a device or within a customer’s infrastructure brings practical constraints that shape the entire system, from hardware selection to deployment and maintenance.
Choosing where inference runs
Weighing latency, data sensitivity, bandwidth and cost for each part of the system.
Connecting to the environment
Integrating models with cameras, sensors, devices and the application services that use the results.
Fitting the hardware
Working within the CPU, GPU, memory and power available on the target machine.
Latency, throughput and connectivity
Keeping up with the incoming data, and behaving sensibly when the network is slow or gone.
Handling local data
Keeping sensitive data on the machine where possible, and sending only what the application needs.
Operating it over time
Deploying and updating models across many installations, monitoring them, and supporting them.
The projects below involved different parts of this list.
Three projects with different constraints.
Proktor is a JIFFY.ai product that Founding Minds engineered. Its computer vision models run on the worker’s own machine and look for defined security events. Processing video on the worker’s device can reduce the need to transmit raw footage for analysis. What is retained or made available for security review depends on the application’s data-handling controls. The case study describes image anonymization techniques that reduce unnecessary exposure of identifiable imagery.
AutoATOps reads the camera feeds an airport already has and records the turnaround events it sees, with timestamps. Airport networks, camera estates and operating rules vary a great deal, and some operators have strict infrastructure requirements, so the system can run on a dedicated on-premises server. It observes and records. Decisions stay with the ground crew.
A pitch is over in a fraction of a second, so BullpenAI pairs a purpose-built high-speed camera setup with computer vision, and a radar gun for velocity, to measure each pitch during training. It shows how much the capture and processing requirements shape the whole system.
What you gain, and what you take on.
Local processing can give you
Less dependence on the network, less data sent elsewhere, and more predictable response times.
It also gives you work to do
Hardware that varies from one site to the next, tight limits on memory and power, model and software updates across many installations, monitoring spread over many places, endpoint security and physical access, and machines that lose connectivity from time to time.
When the cloud is the better answer
Cloud processing suits workloads that need elastic compute, centralized resources, or models too demanding for the available local hardware. We weigh those advantages against latency, connectivity, data-handling requirements, and operating cost before deciding where processing belongs.
Where data is processed shapes what has to move.
Processing data where it is created can reduce what has to be sent, stored or shown to anyone. That helps privacy-sensitive designs, which is how we think about Proktor: decide what the system should detect, and keep everything else out of scope.
Architecture is only part of the answer. Privacy also depends on access controls, how long data is kept, how the application behaves, and how the system is run. A local deployment is not secure or compliant just because it is local.
A detection is only useful once something uses it.
That connection to the application is where much of the engineering effort goes, whether the model runs on a laptop, beside a camera or in a server room.
Illustrative architecture. It is not the design of Proktor, AutoATOps or BullpenAI.
How this fits with the rest.
Computer vision and local inference often go together, but they are separate skills. Computer vision is about detecting, tracking and measuring what a camera sees. This page is about where the inference runs and how the whole system is built for its environment.
Computer Vision →Application Modernization
Bring AI into established systems that have on-premises infrastructure, regulatory constraints, local data or specialized equipment.
Application Modernization →MVP in a Box
Design new AI-enabled products around realistic device, infrastructure, connectivity and operating requirements from the start.
MVP in a Box →Have an AI engineering challenge?
Whether you’re modernizing an existing application or engineering a new AI-native product, we can help determine the right approach.