View profile

Top 10 Machine Learning and Design Insights from Google IO - Issue #3

Victor Dibia
Victor Dibia
Google IO is Google’s annual developer conference where they announce new products, projects and developer tools. I attended several of the sessions and this newsletter contains a list of my top 10 insights that border on Machine Learning and HCI/Design.

LaMBDA - Open Domain Conversational Agent
Google's Language Model for Dialogue Applications. Photo, Google.
Google's Language Model for Dialogue Applications. Photo, Google.
Google is experimenting with language models for open domain conversation - Language Model for Dialogue Applications aka LaMBDA. This is interesting as open domain conversation is is notoriously hard 1 . LAMBDA parses existing data corpuses on multiple domains and generates responses. Once the rough edges are smoothed out (its still in research phase and has quirks) Google hopes to apply it to products like Google Assistant, Search, Workspaces, making them accessible and easier to use.
Building open domain conversational agents - agents that can hold human level conversations for extended turns, on any topic without degrading - ** is notoriously hard **. Such an agent must continually learn, provide engaging content, and stay well-behaved . As Google mentions, its still a bit early and LaMBDA still makes mistakes in its generated responses - logic mistakes (e.g. playing with the moon), factually incorrect responses etc. However this appears promising, especially its applications.
Multi Modal Search
Multi modal models embed data across multiple modalities (text across multiple languages, images, speech, videos, routes etc) into a shared semantic space and enable new use cases. Photo, Google.
Multi modal models embed data across multiple modalities (text across multiple languages, images, speech, videos, routes etc) into a shared semantic space and enable new use cases. Photo, Google.
With neural networks, an interesting trend is the use of multi modal models that can embed data across multiple modalities (text across multiple languages, images, speech, videos, routes etc) into a shared semantic space. If we can do this, we can enable interesting usecases such as cross modal retrieval (e.g. show me all parts of a video that are related to this text query). Or as Google mentions multi modal models allow new types of interesting queries - “show me a route with beautiful mountain views” or “show me the part in the video where the lion roars at sunset”.
Google talked about an implementation - Multitask Unified Model (MUM) based on a transformer model but 1000x more powerful than BERT. MUM can acquire deep knowledge of the world, generate language, is trained on 75+ languages and understands multiple modalities
I really believe explorations in this sort of multi modal representation learning will become the standard for building intelligent recommendation systems. In theory, it lets you learn about the world by modelling joint spaces (e.g. two brands might be equivalent, two parks might be comparable, some things are changeable/moveable while others aren’t .. etc) and it enables you use learned representations for new use cases (e.g. conversational retrieval across multi modal datasets).
Tensorflow.js and Tensorflow Lite - Improved Performance, TFLite Integration
Tensorflow lite models can now run in Javascript. Photo, Google.
Tensorflow lite models can now run in Javascript. Photo, Google.
I have worked with Tensorflow.js extensively in the past and I am quite excited about the recent updates.
  • Tensorflow lite models will now run directly in the browser. With this option, you can unify your mobile and web ML development with a single stack.
  • Tensorflow lite is bundled with Google Play Services which means your Android app apk file is smaller.
  • Profiling tools. TensorFlow Lite includes built-in support for Systrace, integrating seamlessly with perfetto for Android 10.
  • Model Optimization Toolkit The TensorFlow Model Optimization Toolkit is a suite of tools for optimizing ML models for deployment and execution.
  • MLKit . ML Kit brings Google’s machine learning expertise to mobile developers in a powerful and easy-to-use package. Make your iOS and Android apps more engaging, personalized, and helpful with solutions that are optimized to run on device. MLKit (for Android and iOS) now allows you to host your model such that they can be updated without updating your app!
ML and Health
Using AI to help find answers to common skin conditions. Photo, Google.
Using AI to help find answers to common skin conditions. Photo, Google.
There were several interesting ML and Health Applications.
Vertex AI
Vertex AI allows companies to accelerate the deployment and maintenance of artificial intelligence (AI) models. Photo, Google.
Vertex AI allows companies to accelerate the deployment and maintenance of artificial intelligence (AI) models. Photo, Google.
Google introduced a new product - Vertex AI and describe it as a managed machine learning (ML) platform that allows companies to accelerate the deployment and maintenance of artificial intelligence (AI) models. The goal here seems to be focused on streamlining tasks within the AI product lifecycle (which have been previously scattered across multiple GCP services), with special focus on ML operations (MLOps).
They claim that Vertex AI requires nearly 80% fewer lines of code to train a model versus competitive platforms, enabling data scientists and ML engineers across all levels of expertise the ability to Deploy more, useful AI applications, faster.
Given that the MLOps space is still fairly young, I am curious to see how the space gets standardized. As it stands, there are many many different ways to run ML pipeline tasks (e.g. bare metal compute engine instances, managed kubernetes, tensorflow extended, kubeflow pipelines, GCP AI platform etc) even on GCP and it can be confusing for new users.
And more ...
There were more announcements that touched on
  • Automated Design Palettes/Elements for Android
  • Giving users ML and Making it Controllable
  • Privacy - Android Privacy, Privacy Sandbox, Differential Privacy Library
  • Strides in Accessibility, Inclusivity and Responsible AI
  • Wearables - Google WearOS + Samsung Tizen + Fitbit
See the full blog post for more. https://victordibia.com/blog/google-io-2021/.
Did you enjoy this issue? Yes No
Victor Dibia
Victor Dibia @vykthur

Some organized thoughts on research, code and topics at the intersection of applied machine learning and human computer interaction.
Posts by Victor Dibia | @vykthur

In order to unsubscribe, click here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Created with Revue by Twitter.