TWIL: March 12, 2023

Generative AI continues to dominate headlines as companies start leveraging it for a set of diverse use cases. My focus this week was still on Artificial Intelligence topics, like deep learning and model performance evaluation techniques. Also, several news related to AI like the launch of the ChatGPT model on Azure Open AI Service or the Florence foundation model. Finally, I dedicated some time to learn about Microsoft Purview’s lineage features. Enjoy!


Podcasts

.NET Rocks

Episode 1833: MAUI Panel Discussion from Swetugg!
MAUI is out! What’s next? While at Swetugg in Stockholm, Carl and Richard chatted with David Ortinau and Maddy Montaquila about getting MAUI shipped and seeing how the world is using it! The conversation digs into what typical developers are doing with MAUI and other platforms in the plans for future MAUI. David and Maddy talk about how Blazor Hybrid is creating a lot of excitement for folks, balancing the reach of web development with the power of native development – it’s a great time to be building cross-platform software!


Artificial Intelligence

Deep learning vs. machine learning in Azure Machine Learning
This article explains deep learning vs. machine learning and how they fit into the broader category of artificial intelligence. Learn about deep learning solutions you can build on Azure Machine Learning, such as fraud detection, voice and facial recognition, sentiment analysis, and time series forecasting.


AI: Performance Evaluation

An intro to ROUGE, and how to use it to evaluate summaries
ROUGE stands for Recall-Oriented Understudy for Gisting Evaluation. It is essentially a set of metrics for evaluating automatic summarization of texts as well as machine translations. It works by comparing an automatically produced summary or translation against a set of reference summaries (typically human-produced).

The Ultimate Performance Metric in NLP
How do we measure the accuracy of a language-based sequence when dealing with language summarization or translation? For this, we can use Recall-Oriented Understudy for Gisting Evaluation (ROUGE). Fortunately, the name is deceptively complicated — it’s incredibly easy to understand, and even easier to implement.

A Gentle Introduction to Calculating the BLEU Score for Text in Python
BLEU, or the Bilingual Evaluation Understudy, is a score for comparing a candidate translation of text to one or more reference translations. Although developed for translation, it can be used to evaluate text generated for a suite of natural language processing tasks. In this tutorial, you will discover the BLEU score for evaluating and scoring candidate text using the NLTK library in Python.

Generative Model Evaluation
Generative models are notoriously hard to evaluate with traditional metrics. One new way of evaluating them is using language models themselves to do the evaluation. LangChain provides some prompts/chains for assisting in this. The examples here all highlight how to use language models to assist in evaluation of themselves.


Azure Open AI

What’s new in Azure OpenAI Service
Check this webpage for the new features released monthly in Azure Open AI Service.

Learn how to work with Chat Markup Language (preview)
The ChatGPT model (gpt-35-turbo) is a language model designed for conversational interfaces and the model behaves differently than previous GPT-3 models. Previous models were text-in and text-out, meaning they accepted a prompt string and returned a completion to append to the prompt. However, the ChatGPT model is conversation-in and message-out.

Quickstart: Get started using ChatGPT with Azure OpenAI Service
Quickstart guide on how to use ChatGPT in Azure OpenAI Service, either from the Open AI Studio, in Python code or REST API calls.

Revolutionize your Enterprise Data with ChatGPT: Next-gen Apps w/ Azure OpenAI and Cognitive Search
In the context of enterprise applications, the question we hear most often is “how do I build something like ChatGPT that uses my own data as the basis for its responses?”. The combination of Azure Cognitive Search and Azure OpenAI Service yields an effective solution for this scenario. It integrates the enterprise-grade characteristics of Azure, the ability of Cognitive Search to index, understand and retrieve the right pieces of your own data across large knowledge bases, and ChatGPT’s impressive capability for interacting in natural language to answer questions or take turns in a conversation.

ChatGPT + Enterprise data with Azure OpenAI and Cognitive Search
This sample demonstrates a few approaches for creating ChatGPT-like experiences over your own data using the Retrieval Augmented Generation pattern. It uses Azure OpenAI Service to access the ChatGPT model (gpt-35-turbo), and Azure Cognitive Search for data indexing and retrieval.

Introduction to Azure OpenAI Service
Get to know the connection between artificial intelligence (AI), Responsible AI, and text, code, and image generation. Understand how you can use Azure OpenAI to build solutions against AI models within Azure.


Microsoft Purview

Data lineage in Microsoft Purview
This article provides an overview of data lineage in Microsoft Purview Data Catalog. It also details how data systems can integrate with the catalog to capture lineage of data. Microsoft Purview can capture lineage for data in different parts of your organization’s data estate, and at different levels of preparation including raw data staged from various platforms, transformed and prepared data and data used by visualization platforms.

Microsoft Purview Data Catalog lineage user guide
One of the platform features of Microsoft Purview is the ability to show the lineage between datasets created by data processes. Systems like Data Factory, Data Share, and Power BI capture the lineage of data as it moves. Custom lineage reporting is also supported via Atlas hooks and REST API.

How to connect Azure Data Factory and Microsoft Purview
This document explains the steps required for connecting an Azure Data Factory account with a Microsoft Purview account to track data lineage. The document also gets into the details of the coverage scope and supported lineage patterns.

How to view Microsoft Purview Data Sharing lineage
This article discusses how to view the lineage of datasets shared using Microsoft Purview Data Sharing. With these tools, users can discover and track lineage of data across boundaries like organizations, departments, and even data centers.

How to get lineage from Power BI into Microsoft Purview
This article elaborates on the data lineage aspects of Power BI source in Microsoft Purview. The prerequisite to see data lineage in Microsoft Purview for Power BI is to scan your Power BI.

How to use Apache Atlas connector to collect Spark lineage
Apache Atlas Spark Connector is a hook to track Spark SQL/DataFrame data movements and push metadata changes to Microsoft Purview Atlas endpoint.

How to get lineage from Azure Synapse Analytics into Microsoft Purview
This document explains the steps required for connecting an Azure Synapse workspace with a Microsoft Purview account to track data lineage. The document also gets into the details of the coverage scope and supported lineage capabilities.


News

Microsoft’s Latest AI Assistant Is Meant for Marketers, Customer Reps and Work Apps
The software giant is introducing an AI assistant—called Dynamics 365 Copilot —  for applications that handle tasks such as sales, marketing and customer service. Based on technology from OpenAI, the software can draft contextual chat and email answers to customer-service queries. It can help marketers come up with customer categories to target, and write product listings for e-commerce.

Announcing a renaissance in computer vision AI with Microsoft’s Florence foundation model
We are pleased to announce the public preview of Microsoft’s Florence foundation model, trained with billions of text-image pairs and integrated as cost-effective, production-ready computer vision services in Azure Cognitive Service for Vision. The improved Vision Services enables developers to create cutting-edge, market-ready, responsible computer vision applications across various industries.

ChatGPT is now available in Azure OpenAI Service
Since ChatGPT was introduced late last year, we’ve seen a variety of scenarios it can be used for, such as summarizing content, generating suggested email copy, and even helping with software programming questions. Now with ChatGPT in preview in Azure OpenAI Service, developers can integrate custom AI-powered experiences directly into their own applications, including enhancing existing bots to handle unexpected questions, recapping call center conversations to enable faster customer support resolutions, creating new ad copy with personalized offers, automating claims processing, and more.


Have a great week!