GenML 2024

Discover how Amazon Bedrock empowers you to build, evaluate, and deploy enterprise-grade generative AI applications by moving quickly from concept to production. 

This session showcases a real-world example of building an Agentic application using best-in-class foundation models from Anthropic and others, leveraging powerful Bedrock building blocks for RAG, Agents, prompt flows, and Guardrails. 

As a bonus, we’ll share the most exciting generative AI innovations announced at AWS re:invent 2024 (Dec 2-6).

GenAI is a powerful technology that is being rapidly adopted for a plethora of use cases. However, its power also makes it a prime target for attackers looking to exploit vulnerabilities. In this talk, we will discuss the need for GenAI security and the challenges involved in defending against GenAI attacks. We will highlight the complexity of the threat landscape and the difficulties in threat detection in the context of constantly evolving use cases. Join us as we explore this unexplored and uncharted domain and discuss our efforts to break new ground at Intuit with GenOS and GenSRF.

Prompt Flow is an open-source development tool designed to streamline the entire development cycle of AI applications powered by LLMs. Prompt Flow simplifies the process of prototyping, experimenting, iterating, and deploying your AI applications, and has its own VSCode extension.
In this session I'll introduce Prompt Flow and demonstrate how you can use it for building a chat flow, evaluate it and use it as an LLMOps tool.

In this presentation, you'll explore how Weights & Biases are transforming the landscape of machine learning and generative AI. Key takeaways include:

• A technical overview and live demo of the Weights & Biases platform
• Insights into why leading AI teams trust Weights & Biases to scale and productionize ML and GenAI projects
• Real-world case studies showcasing the power of a seamless approach to AI-driven solutions

Traditional LLMs have limitations. They can't access up-to-date information or interact with the real world meaningfully. Function calling solves these problems. This session explores how function calling expands the capabilities of LLMs.

How can we understand the core interests and passions of our buyers? How can we use this knowledge to introduce them to products they have yet to discover?

While typical recommendations often focus on items similar to those already viewed, our goal is to identify the deeper passions that drive buyers' shopping habits. This allows us to suggest related products from entirely different categories.

We use graphs generated by Large Language Models (LLMs) to represent buyers' interests. These graphs help us distinguish between different user interests and traverse the graph to identify various levels of interest granularity.

What are AI agents? What are LLM based agents? How do they differ from non-agentic AI products? What real-world problems do they solve? What are the essential components for developing an LLM based agent? What is the technology stack in this field? How can we evaluate LLM based agents? In this session, we will explore these questions and more. We will examine the architecture, capabilities, and practical applications – Aiming to gain a comprehensive understanding as we create agentic workflows within our organizations.

Talking about GenAI agents is easy, but building ones that actually work is very hard.

In this talk I will shorty introduce what GenAI agents are and will talk about the three biggest challenges in implementing them:

Accuracy and user trust, latency and cost.

I will share a general framework for building GenAI agents using different techniques including different building blocks, planning and methods of evaluation.

I will also discuss how we, at AI21 labs are using these methods to deliver agent based products that actually work

Retrieval-augmented generation (RAG) implementations are becoming increasingly popular, and there is a growing demand for these systems to operate at near-realtime speeds and at scale.

In this session, we present our approach to meet these challenges by bringing the algorithms closer to the data. This involves integrating vector search capabilities with popular databases, enabling RAG implementations on production data.

Specifically, we discuss the necessity of vector data types and the development of advanced index solutions, such as graph-based or inverted-file vector indices, to handle large-scale data efficiently.

By incorporating these features into existing database systems, we aim to unlock the potential for generative AI applications on a grand scale. This session will explore the current state of vector search capabilities, RAG implementations, and other generative AI-related features in a leading database system, highlighting the opportunities and challenges of deploying near-realtime RAG on production data.

Compute power and modern GPU infrastructure are at the heart of the recent GenAI revolution. Companies, research institutes and countries consider it as a strategic enabler for their future success in AI. In this talk we’ll delve into the AI Infrastructure space and share challenges and best practices relating to compute orchestration and utilization.

As large language models (LLMs) become increasingly integrated into industry workflows, a pressing question arises: how do we know if our LLMs are truly delivering value across different user interactions?

This talk, drawing on lessons learned at Lightricks, explores real-world challenges in evaluating LLMs at scale, particularly for creativity-related use cases where there is no single correct answer. We’ll dive into the challenges we faced and share the evaluation framework we developed to align with specific business objectives and user experiences. Attendees will gain practical insights into measuring LLM effectiveness in dynamic, real-world environments, ensuring models are efficient and impactful for their intended use cases.

In this talk, I’ll present how we developed an AI-powered tool at HoneyBook that automatically generates tailored meeting summaries for members. The tool analyzes key data points, including previous meeting notes, client context, shared files, and business insights, to provide members with actionable snapshots before each meeting. This enables them to be better prepared, engage more effectively with clients, and make data-driven decisions. Attendees will learn how AI can enhance productivity through personalized, context-rich insights, and I will explore the technical implementation, challenges faced and results

AI and LLM are in the world on a vast scale. But you can take ML model and LLM to a tiny scale – to the IoT world! You can run models even on tiny microprocessors that cost $4. In this session, you will learn how to work with LLM and ML models on small IoT devices to create exciting and smart devices with low cost and effort

Language models struggle with handling numerical data and performing arithmetic operations. We hypothesize that this limitation can be partially attributed to non-intuitive textual numbers representation. When a digit is read or generated by a causal language model it does not know its place value (e.g. thousands vs. hundreds) until the entire number is processed. To address this issue, we propose a simple adjustment to how numbers are represented by including the count of digits before each number. For instance, instead of "42", we suggest using "{2:42}" as the new format. This approach, which we term NumeroLogic, offers an added advantage in number generation by serving as a Chain of Thought (CoT). By requiring the model to consider the number of digits first, it enhances the reasoning process before generating the actual number. We use arithmetic tasks to demonstrate the effectiveness of the NumeroLogic formatting. We further demonstrate NumeroLogic applicability to general natural language modeling, improving language understanding performance in the MMLU benchmark.