Software Engineer AI Model Services - Level 1

Job Description

Kryptauri is seeking a Software Engineer to help deliver dependable access to large language models. You will obtain, configure, and test models, then prepare them for use through production inference services.

You will build supporting services, improve model delivery pipelines, and work with vendors to integrate proprietary models reliably. Collaboration with infrastructure and application engineers will help connect model services to user tools. The role also includes responding to urgent requests for inference capacity and adapting services as new models and mission needs emerge.

Required Skills

• Three years of relevant experience and a bachelor's degree in a technical discipline; four additional years of experience may substitute for the degree.
• Experience programming in Python or another modern language.
• Experience with Kubernetes and Helm.
• Familiarity with Argo CD or another CI/CD framework.
• Familiarity with AWS or another cloud provider.
• Ability to learn new technologies quickly and communicate clearly with teammates.
• Willingness to ask questions and clarify changing requirements.

Desired Skills

• Experience serving models with vLLM, LiteLLM, or comparable frameworks.
• Familiarity with additional model hosting tools and deployment practices.
• Experience supporting production services through site reliability engineering practices.
• Experience with Elastic, Grafana, Prometheus, or comparable monitoring tools.
• Experience with Docker and containers.
• Experience managing traffic and engineering quality of service.
• Knowledge of and interest in hosting AI capabilities.