Anushrut Gupta, Hasura, Senior Product Manager, Generative AI
“Over the last 3 months, summarize the top billing issues faced by my enterprise customers within the first 30 days of their onboarding.”
On the surface, building an internal AI customer intelligence application that can answer questions like this is a perfect use-case for Gen AI. However, building a production ready app that retrieves the data (RAG) before hauling it off to your favorite LLM for summarization soon becomes a terrible engineering experience.
The data is spread across 3 places: a tickets database (eg: elastic), a CRM (eg: salesforce) and your user-accounts transactional database (eg: postgres). In production, your app can’t access the data from these databases directly. Given security & privacy concerns, your app won’t have direct access to these databases. Making independent retrieval requests to each of these sources and then joining them in memory might be prohibitively expensive and needs a level of query planning to do efficiently. Moving all data into one location is expensive to build, maintain and govern Predictable quality is further made hard because underlying data formats and storage interfaces are continuously changing. Different types of user queries might require additional filtering and joining of data, which becomes hard to generalize.
APIs solve almost all of these very well known challenges. APIs offer standardization and security. APIs can provide a stable contract to interact with underlying data.
And in all likelihood, you already have APIs on these internal and external data sources.
Ironically, while APIs have become a necessity for other parts of the stack, they are clearly not the first thing that AI engineers building RAG reach for.
In this talk, we’ll discuss: Why API based retrieval doesn’t work well for RAG What we need from our existing internal and external APIs to make them RAG ready How we can get existing APIs to become RAG ready without needing to rebuild the APIs
This talk will be technical, with code demos (possibly with some live coding!) and end with key resources (reference architectures, API best practices, tools/technologies) that attendees can take back to their work.
Clemens Vasters is Lead Architect in Microsoft’s Azure Messaging team that builds and operates a fleet of hyper-scale messaging services, including Event Grid, Service Bus, and Event Hubs. Clemens represents Microsoft in messaging standardisation in OASIS (AMQP) and CNCF (CloudEvents... Read More →
Ishneet Dua, Amazon Web Services, Senior Generative AI Solutions Architect Parth Girish Patel, Amazon Web Services, Sr AI/ML Architect
The rapid growth of generative AI brings promising innovation and, at the same time, raises new challenges around its security, safe, and responsible development and use. These challenges include some that were common before generative AI, such as bias and explainability, and new ones unique to generative models, including hallucinations, toxicity, and intellectual property protection. During this session, participants will gain an overview of the challenges that generative AI presents, survey the emerging science surrounding these challenges, and engage in a discussion about the hands-on, security, and Responsible AI work currently being conducted on AWS.
Senior Generative AI Solutions Architect, Amazon Web Services
Ishneet Dua (Isha) is a recognized expert in leveraging AI and machine learning for sustainability solutions. She has established herself as a go-to authority on combating climate change, pollution, and other environmental challenges through cutting-edge technologies.Dua has authored... Read More →
Parth Girish Patel is a seasoned architect with a wealth of experience spanning over 17 years, encompassing management consulting and cloud computing. Currently, at Amazon Web Services (AWS), he specializes in Artificial Intelligence/Machine Learning, generative AI, sustainability... Read More →
Carl Moberg, Avassa, CTO and co-founder Amy Simonson, Avassa, Marketing Manager
Enough manual actions. Enough slow handovers. And enough K8mplexity.
For many innovative enterprises today, the journey to the centralized cloud has shaped the way of working when it comes to container orchestration and observability. Now, developers and IT teams are increasingly also managing containers at the distributed on-site edge and in IoT environments, which risk becoming a mind-boggling task due to the resource-constrained, distant nature of IoT and edge.
In this session, we address the challenges related to deploying, monitoring, observing, and securing container applications at the edge. We also present hands-on examples of what a self-service developer experience can look like for the container applications at the distributed edge and IoT infrastructure. It's automated, it's application-centric and it's astonishingly easy.
Carl has spent many years solving for automation and orchestration. He started building customer service platforms for ISPs back when people used dial-up for their online activities. He then moved on to focus on making multi-vendor networks programmable through model-driven architectures... Read More →
Amy is an experienced marketing professional who thrives right in the intersection between deep tech and marketing. She is currently the marketing manager of Swedish Edge Platform provider Avassa, who are set out to make the distributed on-site edge delightfully easy to manage.
In this session I will guide you from how to start with a CoPilot to deploy and own Azure Open AI instance to use cases which brings benefit to your company. We will also have a look how to build custom solution with the power of Azure.
- Takeaways: - What is a copilot and how you can use it in your daily business - How to setup Azure Open AI - How can you build a custom solution with help of Azure.
My name is Jannik Reinhard and I'm 25 years old and I am work in the internal IT department of the largest chemical company in the world. I am a senior solution architect in the area of modern device management and technical lead of AIOPS (AI of IT Operation).
Timothy Spann, Zilliz, Principal Developer Advocate
In this talk I walk through various use cases where bringing real-time data to LLM solves some interesting problems.
In one case we use Apache NiFi to provide a live chat between a person in Slack and several LLM models all orchestrated via NiFi and Kafka. In another case NiFi ingests live travel data and feeds it to HuggingFace and OLLAMA LLM models for summarization. I also do live chatbot. We also augment LLM prompts and results with live data streams. All with ASF projects. I call this pattern FLaNK AI.
Tim Spann is the Principal Developer Advocate for Data in Motion @ Zilliz. Tim has over a decade of experience with the IoT, big data, distributed computing, streaming technologies, and Java programming. Previously, he was a Developer Advocate at StreamNative, Principal Field Engineer... Read More →
Aman Sardana, Discover Financial Services, Senior Principal Application Architect
Have you ever wondered what it takes to create resilient and highly available platform services that support mission-critical software systems? Please join me to find out how you can set the right strategy and foundational architecture for building platform services that businesses can trust for their most critical workloads.
Payment systems that support real-time transaction processing are expected to be highly available and highly responsive 24/7/365. These systems must be fault-tolerant and resilient to any failures that might happen during payment transaction processing. Mission-critical payment systems with distributed architecture often depend on platform services like distributed caching, messaging, event streaming, databases, etc. that should be independently designed for high availability and fault tolerance. In this talk, I’ll share the approach we took for architecting and designing platform services within the payments domain that can be applied to any domain that supports business-critical processes. This methodological approach starts with establishing a capability view for platform services and then defining the implementation and physical views. You’ll also gain an understanding of other aspects of platform services like provisioning, security, observability, testing, and automation that are important for creating a well-rounded platform strategy supporting business-critical systems.
Senior Principal Application Architect, Discover Financial Services
I am a technology professional working in the financial services and payments domain. I’m a hands-on technology leader, enabling business capabilities by implementing cutting-edge, modernized technology solutions. I am skilled in designing, developing, and implementing innovative... Read More →
Context is fundamental to well-run tech operations: With the right context, IT teams can better understand their systems, interpret real-time data quickly, and facilitate better incident management to achieve operational efficiency. But too often, gathering the necessary context is a lengthy, inconsistent, and elusive process. IT teams are forced to grapple with fragmented tools, siloed workflows, and inconsistent manual processes, which have turned context collection into a definitive pain point for the ITOps industry. Teams are losing out on precious time, money, and attention that should be directed towards digital transformation and innovation.
The tech industry has recently transformed thanks to the AI boom: ITOps is at a critical juncture where AI can enable faster, more efficient ITOps as well as deliver Full-Context Operations. Fred Koopmans, Chief Product Officer of AIOps platform BigPanda, will speak to the promise of Full-Context Operations – the process of unifying IT teams’ tools and processes with AI to provide the institutional knowledge needed to address every incident immediately. He’ll dive deep into the ways that teams can tangibly benefit from having the right context, outlining how the IT industry can leverage AI to collect comprehensive and contextual data to help operators achieve better incident resolution. Fred can share detailed proof points from developing BigPanda’s AI-powered assistant that was purpose-built for delivering full context in IT operations. With Full-Context Operations, the IT industry can finally fulfill the long-sought-after promise of AIOps, putting AI into practice to deliver unprecedented operational efficiency.
Fred Koopmans, BigPanda's Chief Product Officer, is dedicated to driving innovation and collaboration, building trusted partnerships with customers, creating product roadmaps, and empowering individuals to achieve the extraordinary. He leads product strategy, product management, product... Read More →
