From POC to Production: The Infrastructure for Gen AI at Scale