I had two weeks of vacations, but the technology world never stops, and there is always something new to learn. This week I’m highlighting a great episode of the AI Portugal Podcast with Manuel Dias (Microsoft Portugal’s National Technology Officer) on Generative AI and Microsoft’s role in this new revolution. Also, a ton of news related with Large Language Models (LLMs) such as the Skeleton-of-Thought paper, OpenAI’s GPTBot and GPT-5 rumours, Gorilla and Tool-LLM models and Llama 2 ONNX. Finally, interesting GitHub repos for LayoutXLM and Vision AI Solution Accelerator, and articles on Azure Cognitive Services Language model lifecyle, Azure Synapse Link for Dataverse, Azure Cosmos DB for MongoDB and Azure Cognitive Search with Langchain. Have fun!

---

Podcasts

Building the Future AI Portugal Podcast

Portuguese podcast born out of the annual Building the Future event where technologies, ideas and initiatives that transform our world are discussed, with a particular focus on Artificial Intelligence. The episodes of this podcast are spoken in portuguese, as are their descriptions.

Generative AI + OpenAI + Azure
Sobre o nosso episódio de hoje, já temos abordado imenso o tópico do chat GPT modelos GPT 3, 3.5 e 4, e falado sobre toda esta revolução que temos tido no mundo com a introdução desta tecnologia e o seu crescimento em termos de use cases práticos e produtização. Mas a verdade é que ainda não falamos em detalhe dessa parte em particular, a produzição deste tipo de modelos de forma a empresas construírem realmente soluções enterprise grade sobre modelos como estes, e em particular sobre a parceria que a Microsoft fez com a OpenAI neste ponto de tornar disponível todo o potencial destes modelos no Azure com um monte de features adicionais de governo e segurança que tornam este serviço mais adequado a enterprise customers. Vamos falar sobre use cases, vantagens destes modelos e o que significa usar OpenAI no Azure e desbravar terreno no que são os usecases de AI para grandes empresas e as implicações dessas arquiteturas mais complexas.

---

Cloud Architecture

Empowering AI: Building and Deploying Azure AI Landing Zones with Terraform
To harness the true potential of AI technologies, like GPT-4, a robust and efficient infrastructure is crucial. Azure Landing Zones provide a structured approach to create a rock-solid cloud environment, while Azure OpenAI Service seamlessly handles AI model deployment and management.

---

Large Language Models

Skeleton-of-Thought: Large Language Models Can Do Parallel Decoding
This work aims at decreasing the end-to-end generation latency of large language models (LLMs). One of the major causes of the high generation latency is the sequential decoding approach adopted by almost all state-of-the-art LLMs. In this work, motivated by the thinking and writing process of humans, we propose “Skeleton-of-Thought” (SoT), which guides LLMs to first generate the skeleton of the answer, and then conducts parallel API calls or batched decoding to complete the contents of each skeleton point in parallel. Not only does SoT provide considerable speed-up (up to 2.39× across 11 different LLMs), but it can also potentially improve the answer quality on several question categories in terms of diversity and relevance. SoT is an initial attempt at data-centric optimization for efficiency, and reveal the potential of pushing LLMs to think more like a human for answer quality.

GPT-5 is Official now! + OpenAI Debuts GPTBot
OpenAI’s GPT-5 and GPTBot are poised to transform the AI landscape, with GPT-5 possibly setting new records in size, accuracy, and versatility. GPTBot, a web crawler, plays a critical role by gathering high-quality data, adhering to privacy guidelines, and fueling research. This exploration of OpenAI’s innovations, including whether GPT-5 could reach Artificial General Intelligence (AGI), holds great significance for the future of technology, ethics, and various applications across industries.

OpenAI deploys web crawler in preparation for GPT-5
OpenAI has introduced a web crawling tool named “GPTBot,” aimed at bolstering the capabilities of future GPT models. The company says the data amassed through GPTBot could potentially enhance model accuracy and expand its capabilities, marking a significant step in the evolution of AI-powered language models. OpenAI’s GPTBot will have a distinct purpose: to gather publicly available data while carefully sidestepping sources that involve paywalls, personal data collection, or content that contravenes OpenAI’s policies.

GORILLA AI: Meet the First Genuine Proximate AGI (By Microsoft)
Gorilla is an AI model, developed by Microsoft and UC Berkeley, that can autonomously interact with various online tools using API calls. Unlike other large language models, Gorilla can learn from online sources in real-time, adapt to changes, and handle complex tasks spanning different platforms. Explore its capabilities, understand its rise toward artificial general intelligence, and decide if it’s the future of AI or just passing hype.

This AI is 10X More Powerful than GORILLA AI (Proximate AGI)
Tool-LLM is a groundbreaking AI model that excels at working with over 16,000 real-world APIs, making it 10 times more powerful than the Gorilla AI Model. Developed by experts from Meta, Microsoft, Stanford, and UC Berkeley, Tool-LLM can interact with various online services, adapt to new APIs, and is available on GitHub for users and developers. Dive into its capabilities, comparisons, and potential applications in industries like education, healthcare, and finance.

Llama 2 Powered By ONNX
This is an optimized version of the Llama 2 model, available from Meta under the Llama Community License Agreement found on this repository. Microsoft permits you to use, modify, redistribute and create derivatives of Microsoft’s contributions to the optimized version subject to the restrictions and disclaimers of warranty and liability in the Llama Community License agreement.

Generative Agents: Interactive Simulacra of Human Behavior
This repository accompanies our research paper titled “Generative Agents: Interactive Simulacra of Human Behavior.” It contains our core simulation module for generative agents—computational agents that simulate believable human behaviors—and their game environment. Below, we document the steps for setting up the simulation environment on your local machine and for replaying the simulation as a demo animation.

Generative Agents: Interactive Simulacra of Human Behavior
Believable proxies of human behavior can empower interactive applications ranging from immersive environments to rehearsal spaces for interpersonal communication to prototyping tools. In this paper, we introduce generative agents–computational software agents that simulate believable human behavior. Generative agents wake up, cook breakfast, and head to work; artists paint, while authors write; they form opinions, notice each other, and initiate conversations; they remember and reflect on days past as they plan the next day. To enable generative agents, we describe an architecture that extends a large language model to store a complete record of the agent’s experiences using natural language, synthesize those memories over time into higher-level reflections, and retrieve them dynamically to plan behavior.

---

Machine Learning

Vision AI Solution Accelerator
This repository serves as a rich resource offering numerous examples of synthetic image generation, manipulation, and reasoning. Utilizing Azure Machine Learning, Computer Vision, OpenAI, and widely acclaimed open-source frameworks like Stable Diffusion, it equips users with practical insights into the application of these powerful tools in the realm of image processing.

LayoutXLM (Document Foundation Model)
LayoutXLM is a multimodal pre-trained model for multilingual document understanding, which aims to bridge the language barriers for visually-rich document understanding. Experiment results show that it has significantly outperformed the existing SOTA cross-lingual pre-trained models on the XFUND dataset.

---

Azure Cognitive Services

Language Service: Model lifecycle
Language service features utilize AI models. We update the language service with new model versions to improve accuracy, support, and quality. As models become older, they are retired. Use this article for information on that process, and what you can expect for your applications.

---

Azure Cosmos DB

MongoDB on Azure: Managed Service vs. Self-Managed
The MongoDB database provides organizations an opportunity to evolve and store data without being confined to the initial underlying data model. As part of an Azure database deployment, MongoDB can add its capabilities to the Microsoft cloud’s scalability and services. These capabilities can make MongoDB usage a key part of your Azure big data workloads.

What is Azure Cosmos DB for MongoDB?
Azure Cosmos DB for MongoDB makes it easy to use Azure Cosmos DB as if it were a MongoDB database. You can use your existing MongoDB skills and continue to use your favorite MongoDB drivers, SDKs, and tools by pointing your application to the connection string for your account using the API for MongoDB.

---

Azure Synapse Link

Accelerate time to insight with Azure Synapse Link for Dataverse
With a few clicks, you can bring your Dataverse data to Azure Synapse, visualize data in your Azure Synapse workspace, and rapidly start processing the data to discover insights using advanced analytics capabilities for serverless data lake exploration, code-free data integration, data flows for extract, transform, load (ETL) pipelines, and optimized Apache Spark for big data analytics.

Power your business applications data with analytical and predictive insights
The data within Dataverse is a goldmine of potential insights that analytics can easily bring to the surface. With Azure Synapse Link for Dataverse, customers can now automatically ensure that data flowing into their business applications is also flowing into their analytics solution. This enables customers to perform advanced analytics tasks in tandem with managing the data in their business applications—rather than having these be separate workstreams.

---

Azure Cognitive Search

Azure Cognitive Search and LangChain: A Seamless Integration for Enhanced Vector Search Capabilities
This article explains how to use vector search in Azure Cognitive Search, a feature that allows you to find relevant results based on natural language understanding. It also introduces LangChain, a framework for developing applications powered by language models, and how it can be integrated with Cognitive Search to perform similarity search, hybrid search, and semantic search. The document provides code samples and commands for setting up the OpenAI settings, creating the vector store, chunking and vectorizing documents, loading the index, and executing different types of search queries. The document also compares the advantages and disadvantages of different search modalities and gives guidance on when to use each approach.

---

Have a great week!