Teaching Ozlo about Pokémon GO

Anant Narayanan's blog - jeu, 04/08/2016 - 01:00

Pokémon GO is all the rage these days. Ozlo, your friendly AI sidekick, would be remiss if he didn’t help you catch them all!

Thanks to Ozlo’s unique, knowledge-based approach to the world, we were able to teach him about Pokémon in just under a week, including how to find PokéStops and Pokémon Gyms near places you might be going. In this blog post, we’ll take a look at some of Ozlo’s inner workings, what goes into teaching him a completely new concept, and why his ability to learn quickly matters.

The process involves three high-level steps:

  • Feeding Ozlo data about the new concept
  • Teaching Ozlo to understand how people talk about the concept
  • Teaching Ozlo how to talk to people about what it knows

We’ll cover each of these steps one-by-one and then discuss why it’s important we do things this way — and why that makes Ozlo fundamentally different than many other chatbots and AI assistants out there.


Ozlo’s view of the world consists of entities (people, places, or things) and relationships among them. Teaching Ozlo about something new begins with acquiring data about the subject that so we can augment his knowledge of the world. This can happen by several means — crawling the web, hitting APIs, and obtaining data from partners, for example.

In Pokémon GO’s case, we decided to focus on a use-case that helps you play the game effectively but doesn’t break it or cheat in any way: finding PokéStops. PokéStops are places all around the world, and they have certain attributes that identify them: coordinates, a name, picture and sometimes a description.

Once we found all the PokéStops in the US, we turned them into entities and started creating relationships. Ozlo already knows about all the cities in the US as well as what landmarks and restaurants exist in each city. With this knowledge, he can perform reasoning to know that if a given place is inside the polygon for a given city’s boundary, then the place must be in the city (and so on…)

When this process concludes, Ozlo has a mental map of where all the PokéStops in the US are located, which of them are “gyms”, what cities they are in, and which landmarks and restaurants they are near.


Next, we had to teach Ozlo some of the common ways in which humans might ask him about PokéStops. In the beginning that involves just writing out some examples and telling Ozlo what each of them mean.

Consider the following sentence, resembling something a human might ask Ozlo:

“Show me pokemon gyms near the ferry building”

There’s a lot in that sentence that Ozlo can already understand! He has a basic understanding of the English language, but also knows how people talk about restaurants and landmarks (since we taught him that earlier). What does Ozlo see in that sentence?

“show me”: Here’s a hint that the answer to this question requires some sort of visual presentation.

“near”: I’ve seen this word many times before and when it is followed by a name of a place, I know what that means.

“ferry building”: Looks like I have many entities that match this name. But, I can rank all the places with this name by their popularity and distance from where the user currently location to narrow down a likely candidate.

The only part of that sentence Ozlo didn’t quite understand was “pokemon gyms”. This is where we step in and give him some examples along with what they mean:

“pokestops”: This means entities that are PokéStops

“pokemon gyms”: This means entities that are PokéStops of type “gym”

We also added many more variations of the above to give him a basic understanding of PokéStops. And don’t forget — Ozlo also keeps learning as you use him — so he’ll collect a lot more examples over time than what we just start him off with!


The final step was to teach Ozlo how to turn his answer into words and interactions that humans can understand. In many ways this is exactly the reverse of Ozlo trying to translate what a human said into terms he can understand.

Ozlo already has a good knowledge of English, so he can mostly construct the sentence on his own. We just need to give him a few hints and we get:

“There are many Pokémon Gyms around Ferry Building”

Then we construct the visual format of the response. In our iOS app we settled on using the “multi-pin map” element, which is an easy way to view several points of interest in a given geographic area. For now, we just tell Ozlo what type of visual result format to use, based on the user’s device.

Ozlo’s capabilities aren’t limited to just rendering maps though - he can choose between a variety of output formats - and we pick the one that’s best suited to the medium you’re using to communicate with him.

Why This Matters

Why go to all this effort to actually teach Ozlo about PokéStops instead of just having Ozlo redirect your question to some other service? We’ve talked about the multi-agent problem before — and we believe there is a fundamental difference between bots that know things and bots that guess what other services might know about things.

As Ozlo’s knowledge of the world grows, adding more data to it enriches his entire world view. There’s a network effect between entities — because these entities have relationships with each other — adding new entities has an exponential effect on Ozlo’s understanding of the world. This is what lets us leverage the fact that Ozlo already knows about “the Ferry Building” to help you find out what PokéStops are near it with only a minimal amount of effort.

We can’t wait for the day where we’re not the only ones teaching Ozlo about new concepts! In the meantime, please keep using Ozlo and giving him feedback to help him continue to learn more about the world.

Catégories: Not PHP-GTK

7 Tips for php[tek]

Ben Ramsey - dim, 22/05/2016 - 01:00
This week, I’m attending php[tek]. This is my seventh php[tek], and the first I’ve attended not as a speaker. It’s one of my favorite conferences, and I didn’t want to miss its first year in a new city: St. Louis. As we gear up for the eleventh php[tek] conference, I thought I’d list my seven tip...
Catégories: Not PHP-GTK

Mastering OAuth 2.0 Published

Ben Ramsey - dim, 15/05/2016 - 03:00
I've published my first article in _php[architect]_ magazine since 2009! It's only fitting that it's an article on OAuth 2.0, since one of the last articles I published in their magazine was on OAuth 1.0. I'm proud and excited to finally publish a new article with them after such a long hiatus, a...
Catégories: Not PHP-GTK

Meet Ozlo

Anant Narayanan's blog - jeu, 12/05/2016 - 01:00

Two days ago, a project I’ve been working on for a little over two years was unveiled to the world. Meet Ozlo, your friendly AI sidekick!

First things first: if you haven’t signed up yet, hit up this link which includes a VIP code to fast-track you into our invite-only app.

A lot has been said about Ozlo already: Charles Jolley (co-founder), John Lilly (investor), Lloyd Hilaiel (friend & colleague), Todd Agulnick (friend & colleague) and even Buzzfeed! Here’s my perspective…


It didn’t take me very long since I first heard the idea for a better mobile search experience from Mike John calls Mike an “anytime, anything, anywhere” person, and it couldn’t be truer. and Charles to stop what I was doing and jump on board.

The fundamental problem we’re trying to solve is that even though our smart phones enable us to do a lot more than we could before, the process of finding people, places and things on them is not very different from how you would do it on a desktop.

That’s usually the natural course for any technology to take. Take publishing for instance – when tablets were first introduced – a publication’s first instinct was to just take what they had on paper and turn it into pixels. The first application on any new platform usually is a v1 – “available here too” – product. This first version often under-utilizes the platform’s true capabilities and its creators can quickly be lulled into thinking that they’ve created the optimal experience for the consumer.

What’s v2 for search on mobile devices? To answer this question is why we created Ozlo.


In attempting to answer this question, we built something that we thought might work. It didn’t work quite as well as we’d have liked. So we did it again. And again. Fast-forward two years and you arrive at Ozlo: a personal and intelligent companion that helps you find things.

The first manifestation of that idea is an iOS application that can help you find food. In the app, you interact with Ozlo via a chat-like interface. Here I am trying to find that place that I can’t quite remember the name of:

This iteration of the app is purposely focused on one goal – finding you food. But there are several underlying themes that have the potential to pave Ozlo’s way to something grander:


Searching for something is usually not a one-shot type of activity. Humans don’t work that way. We ask a question, and often follow up with more questions; until we’ve refined our own thoughts to ultimately get the answer we’re looking for. It’s exciting that Ozlo has the potential to participate in this back-and-forth.

It’s not merely a co-incidence that my previous blog post, a little less than a year ago was about the Amazon Echo. I have an Alexa at home and I love it. I can’t wait for the day where I can talk to Ozlo like I can talk to Alexa, only better!


Ozlo has the potential to know you over time, learn about your preferences and interests in a meaningful way. To me, this brings a face to the otherwise utilitarian search box that feels disconnected and impersonal.

As a vegetarian, I can already appreciate Ozlo helping me find hidden gems at restaurants I’d usually dismiss. What if Ozlo could also recommend movies for me to watch, grab that hard to get restaurant reservation and help me find the perfect anniversary gift?


In the past few years, technology seems like it’s finally getting to the point where building an agent that can really understand what humans say is tantalizingly close to being possible. We’ve observed the resurrection of the term “AI” to refer to this sort of thing. It’s often an overloaded term, but there is no doubt that the industry as a whole has made big technological strides in deep learning and machine intelligence.

Ozlo is different from usual search engines, the ones that return results with the same words as your query, without knowing what the words mean. Ozlo tries to understand what you said and then tries to arrive at an answer. To me, that makes Ozlo intelligent.

Training Ozlo to understand the nuances of human language is going to be a very difficult task. But it is by no means impossible, given the resources we (as computer engineers and scientists) have at our disposal these days.


The really interesting bits are in the technology behind Ozlo and how we built it. This is some of the deepest technology I’ve ever had a part in building and I’m extremely proud of it. To make Ozlo work, we’ve had to write several pieces of software from scratch.

On the backend:

  • Data Pipeline:
    to ingest, dedupe and glean structure from the mess of data we find; at scale; with speed.
  • Search Engine:
    to index the facts our data pipeline emits and allow us to efficiently query it; at scale, with speed.
  • Query Understanding:
    to turn human language into a series of structured queries machines can understand.
  • Dialog System:
    to keep track of the high-level structure of the conversation you’re having with Ozlo.

On the frontend:

  • Language Synthesis:
    to turn structured results back into friendly text humans can understand.
  • Layout Language:
    to efficiently and generatively render results as a graphical layout.
  • View Synthesis:
    to aggregate, refine and generate the final layout humans will see.
  • iOS App:
    to turn that layout back into pixels that are delightful to look at and interact with.

We built most of our backend in Go. It’s no secret that I’ve been a fan of Go since its inception, primarily because of my affinity to Plan 9; but this is the first time I’ve been able to observe it being used at a large scale for a production-quality project. I couldn’t be happier with our choice, and I’ll admit that I’ve had some days where I get into work only because I’m excited by the prospect of writing some Go.

We built most of our frontend in NodeJS (and ObjC for the iOS app, of course). It’s also no secret that I’ve been a huge proponent of Javascript and our frontend has been chugging along happily (we’ve had a few refactorings, but really, what JS code base doesn’t go through atleast two?). Say what you will about JS & NPM, especially in the recent past, one cannot deny the convenience and speed of development that is offered by the JS runtime.

If any part of this sounds exciting to you, why not join me in working on Ozlo?


Launches are really fun. But the best part comes right after. As we see how people use and interact with Ozlo, I can’t wait to see where we take him next. Movies? Music? Sports? Products? Ozlo in the car? On Alexa at home? Most likely something we haven’t thought of yet.

Come be a part of it – don’t forget to use the VIP code to sign up – and please send us your feedback!

Catégories: Not PHP-GTK

Post-Open Source

Ben Ramsey - mar, 26/04/2016 - 01:00
I’m a tad late to this discussion, but I think it’s still pertinent today—perhaps even more so—and Jordi Boggiano’s recent post, “Common files in PHP packages,” got me thinking about the lack of open source licenses in public repositories.
Catégories: Not PHP-GTK

Introducing ramsey/uuid

Ben Ramsey - dim, 24/04/2016 - 19:00
It seems quite absurd for me to introduce ramsey/uuid, a library that saw its 1.0.0 release on July 19, 2012, and is now at version 3.4.1, having had 35 releases since its first, but what’s even more ludicrous is that I haven’t once blogged about this library. I mention it only in passing in my “...
Catégories: Not PHP-GTK
Syndiquer le contenu