Model-as-a-Service
- Access model over internet
- Surface model through web portal
- Users post queries/data and receive predictions
On-Device Serving
- Edge computing
- Helpful for unreliable internet or secure environments
Pros of On-Device Serving:
- Lower latency
- Enhanced security
- Works in remote/disconnected areas
Cons of On-Device Serving:
- Resource constraints
- Challenges with model updates
- Monitoring difficulties
Implementation Strategies:
- Model pruning
- Transfer learning
- Use of dedicated frameworks for edge deployment