06 November 2017

What are chatbots? And what is the chatbot community?

In the beginning, all bots on IRC (Internet Relay Chat) were popularly referred to as “chat bots”.  Basically, IRC was the predecessor of IM (Instant Messaging) for realtime chat.  And Facebook Messenger is basically the successor of IM.  

After years of IM services fighting bots and automation, in a surprise move Facebook opened Messenger to bots in April 2016, which I call the “Facebook April surprise”.  Immediately, people began referring to Facebook Messenger bots as “chat bots” (note space).  Until then, the term chatbots (no space) had been gradually taking over the space previously known as chatterbots.

Since the Facebook April surprise, there has been a grand confusion reigning with people talking at cross-purposes about chatbots, challenging expectations all around.  Basically, Facebook Messenger chatbots have become “chat apps”, with lots of graphical UI elements, such as cards, interspersed with natural language.

Prior to the Facebook April surprise, there has long been a robust chatterbot community largely gathered around the controversial Loebner Prize.  Until today, the Loebner Prize has been the only significant implementation of the Turing test in popular use.  I happen to believe that the Turing test itself is problematic, if not a red herring; however, the contest’s founder Hugh Loebner deserves a place in history for stimulating the art, especially through the so-called AI winter.

There are further stakeholders in this melee.  In addition to the academic community of artificial intelligence researchers, there is also the natural language processing community.  Some people count NLP as a subset of AI, though a good argument can also be made against that.  My long investigation into NLP has shown to me that natural language processing has been largely predicated on the analysis, or deconstruction, of natural languages, for instance in machine translation, leading to natural language understanding.  It is only relatively recently that an emphasis has been placed on the construction of natural language, generally referred to as natural language generation.

Artificial intelligence itself is not a very useful term, as it implies replicating, or copying, human intelligence, which carries its own set of baggage.  As used today, it is so broad as to be ineffectual.  In short, AI researchers are not necessarily chatterbot, or dialog system, researchers, and nor are NLP researchers.  There are various and sundry loci for high level discourse on dialog systems for the academic community, often with large corporations hanging around the periphery.

There used to be a very good, informal mailing list for the Loebner Prize crew, but it suddenly got deleted in a fit of passion.  From there, the chatterbot community more or less came in from the cold of perhaps a dozen separate web forums gradually to the chatbots.org “AI Zone” forum, largely dedicated to the art of hand-crafting so-called pattern-matching dialog systems, or chatterbots.  

Hot on the heels of the Facebook April surprise, an enterprising young man named Matt Schlicht opened the Facebook Bots (chatbot) group, which had close to 18,000 members six months later (and close to 30,000 members today, 18 months later).  I would say throughout that process it has provided an informative and dynamic timeline, around which a new community has rallied.  However, that same community has collectively come to realize that Facebook Messenger “chat apps” are not the chatterbots everyone has been dreaming about.

Matt Schlicht had a proverbial tiger by the tail, in the form of his Facebook Bots group.  Due to the public pressure of a scandal of his own making, he initiated the process of electing a moderation team in November 2016.  I know how difficult this can be, managing online communities, through my own experience with the once popular travel mailing list, infotec-travel, throughout the dot-com bubble of the 1990s, an online community which ultimately lead to much of the online travel infrastructure we enjoy today.

Not only have I been banned from Matt Schlicht’s Facebook Bots group, but have been banned twice, and am still banned today.  The first time I was banned after posting about my chatbot consulting services.  However, due to the gracious intercession of current Loebner Prize holder, Mitsuku developer Steve Worswick, my group membership was reinstated.  I was then banned for the second time after sharing a private offer from Tinman Systems for their high end artificial intelligence middleware.  

After being ejected from the Facebook Bots group for the second time, I started my own Facebook group at Chatbot Developers Bangalore, due to my particular interest in AI, NLP, and chatbots in India.  (I also happen to be co-organizer of the Bangalore Robotics Meetup.)  Today, I blog about this a year later, because one of those new Facebook Bots group admins stirred the controversy by requesting admission to a closed Facebook group of which I'm a member, Australian Chatbot Community.

This blog post was originally submitted to VentureBeat in November 2016, prior to the successful election of a new administration team for the Facebook Bots group.

07 March 2017

Marcus Endicott successfully predicts IBM Watson Salesforce partnership

IBM Watson announced partnership with Salesforce Einstein March 06, 2017.

Mar 06, 2017: IBM and Salesforce today announced a global strategic partnership to deliver joint solutions designed to leverage artificial intelligence and enable companies to make smarter decisions, faster than ever before. With the partnership, IBM Watson, the leading AI platform for business, and Salesforce Einstein, AI that powers the world’s #1 CRM, will seamlessly connect to enable an entirely new level of intelligent customer engagement across sales, service, marketing, commerce and more. IBM is also strategically investing in its Global Business Services capabilities for Salesforce with a new practice to help clients rapidly deploy the combined IBM Watson and Salesforce Einstein capabilities.

Salesforce Einstein was launched in September 2016.

Sep 18, 2016: Salesforce forms research group, launches Einstein A.I. platform that works with Sales Cloud, Marketing Cloud

For those still paying attention... I have been going on and on about this needing to happen for the past three years, on Quora.

Sep 1, 2014: I don't know of another system that integrates more systems, more easily than Salesforce. My main critique of Salesforce is that it is too rigidly focused on conventional business process, and does not allow enough leeway for the Internet of Things, much less for experimental AI....

Nov 13, 2014: Bluemix appears to be an empowerment play to widen the base of developers to include those less proficient in pure coding, along the lines of Salesforce. That said, when Bluemix becomes as user-friendly as Salesforce, only then will I consider it fully baked.

Feb 24, 2015: There needs to be something along the lines of Salesforce that is not exclusively limited to conventional business processes, but something broad enough to include all the possibilities of experimental AI.

Dec 3, 2015: I'm most interested in "Lego-ization", and the plug-and-play model, which to some degree would require as yet non-existent standards. Think "Integration Platform as a Service", something along the lines of Salesforce meets MATLAB, up to the challenges of experimental AI of all kinds.

Aug 3, 2016: I want a *visual* middleware, along the lines of the highly modular Salesforce, but for experimental artificial intelligence instead of severely restricted to conventional business solutions.

15 September 2015

2015 PATA Technology Forum, Bangalore

06 September 2015, I had the pleasure of attending the first Pacific Asia Travel Association Technology Forum, at Bangalore International Exhibition Center, in partnership with phocuswright.com and connectingtravel.com. PATA is the travel and tourism industry association for the Asia Pacific region, now headed by artificial intelligence investor Mario Hardy. Phocuswright is the global nexus for technology in travel and tourism. Connecting Travel is a new professional social network initiative for the travel and tourism industry by Travel Weekly. (Both Phocuswright and Travel Weekly are now owned by Northstar Travel Media.)

The opening speaker was the prominent investor and philanthropist Mohandas Pai, in his role as chairman of the Karnataka Tourism Vision Group. Pai, who is heavily invested in tripfactory.com, provided a 360 degree overview of the skyrocketing digital economy in India, as well as its impacts on travel and tourism, both domestically and internationally. One of the most interesting things he mentioned was the Aadhaar, or Unique Identification Authority of India, basically the world's largest national identification number project, set to biometrically empower millions of people without conventional paper trail or fixed abode.

Tony D'Astolfo, managing director of Phocuswright, introduced this new "Phocuswright Fast Track", by calling it an event within an event. Phocuswright offers recent research on the Indian travel market, and not only maintains a dedicated team in India, but also is planning a full Phocuswright India travel technology conference 21-22 April 2016 near New Delhi, in Gurgaon.

Chetan Kapoor, Phocuswright research analyst for Asia Pacific, put the spotlight on Indian holidays and package travelers, highlighting the evolution of the Indian traveler, and how their shopping and booking habits are transforming traditional holidays and packages.

In the first executive roundtable, titled "Beyond Air - The Next Phase of India's Online Travel Story", Chetan Kapoor presented three of India's new travel and tourism heavyweights:
In terms of traffic, Tripadvisor is consistently within the top 3 travel sites in India, listing more than 30,000 Indian accommodations, with the largest number of reviews. HolidayIQ is a Bangalore-based travel information and review portal, with over 3 million members, listing 2,000 tourism destinations, and more than 50,000 accommodations, in India alone. Cleartrip is one of the top online travel agents in India, attracting more than $70 million in funding.

In the second executive roundtable, titled "Travel Innovation Summit Alumni Spotlight", Tony D'Astolfo introduced three of India's most innovative entrepreneurs to discuss how they are transforming the travel industry, at home and abroad:

Intuitive travel planner Mygola has recently been acquired by MakeMyTrip, one of India’s leading travel companies. TableGrabber, India's first real-time online restaurant reservation system, has recently launched RezGuru, a middle-layer software for restaurants. TripHobo, a travel itinerary-planning portal, recently announced a partnership with Zomato, a leading restaurant discovery platform made in India.

For the executive interview, Tony D’Astolfo did a one-on-one with Ritesh Agarwal, 21 year old founder and CEO of OYO Rooms, India's largest branded network of hotels. Not only is he one of India's youngest CEOs, but also India's most successful college drop-out. Agarwal was the first Indian to receive a $100,000 fellowship grant from Peter Thiel, which he invested in developing OYO Rooms. And mostly recently, he has raised $100 million from Japan’s SoftBank for OYO Rooms. Legend has it that Agarwal started OYO Rooms, which stands for "On Your Own", because his relatives would not let him control the TV remote when he was a child in India. On a personal note, I can say for sure that I am staying in better places in India, and paying less, now than I was a year ago, due to the phenomenal concept that is OYO Rooms.

Following lunch, Connecting Travel organized the "Technology Trends Defining Business Strategy" session, moderated by Tony Tenicela, IBM executive and global leader managing business development. This session focused on how global market players are redefining business models to adapt to the accelerated pace of communication, marketing, and loyalty initiatives. Social media, and virtual networks, figure prominently in creating vertical platforms that are aggregating professionals, consumers, advisers and investors into communities.
Helena Egan, director of industry relations at TripAdvisor, is primarily concerned with building relationships with destination marketing organisations, as well as educating the industry on the benefits user-generated content. Kenny Picken, CEO of Traveltek, a leading provider of travel technology solutions, shared valuable insights of how Traveltek empowers industry stakeholders, rather than by-passing them. Philip Napleton, VP at Open Destinations, providing software for tour operators and wholesalers, emerged as the voice of the younger generation, with his insight into social media and mobile applications. Rika Jean-Francois, head of corporate social responsibility for Internationale Tourismus-Boerse Messe Berlin, was the only person to emphasize the potential of travel technology in developing sustainable tourism. Mike Kistner, CEO of RezNext, a real-time hotel distribution technology company, provided perspectives of the seasoned travel technology professional. Daniela Wagner, Connecting Travel at Travel Weekly, spoke of how their new social network platform can benefit travel professionals.

Note, YourStory is the largest platform for news, reports and analysis on India's booming startup ecosystem.

07 December 2013

Taking It On The Road, Travel Technology 2013

I spent much of this year mountaineering in Europe, and re-visiting India after 34 years. Since my father's passing earlier this year, I've been free to travel again. My father was my chief technology influence. People often ask how I got into technology, since my education was in psychology and most of my career was in tourism. It was all due to my father, Lucian J. Endicott Jr.,who worked nearly three decades for IBM, and then became a professor of computer science before retiring altogether.

On this journey I've been watching closely to see what technologies I find most useful. Unlike most of the young people traveling today, I'm not traveling with a phone. I did have an Apple iPod Touch for awhile, which I enjoyed, but passed it on in favor of the new Google Nexus Android tablet. I find phones and tablets great for everything other than real work, like programming. I did buy the most economical, top rated Consumer Reports laptop for students, and have been very happy with it.

A man can only travel with so many devices though; so, the Apple iTouch and Android tablet both went to nieces, and I'm still happily traveling with my affordable laptop. In both Europe and Asia I have found locally available, prepaid "data cards" or "dongles", basically a phone chip on a USB stick, very helpful for freeing myself from dependence on wifi. However, some of the new, higher end phones come with built-in wifi "hotspot" capability, which I've seen quite a few young people using with their laptops. Without a phone per se, Skype has proven super convenient, especially premium Skype to local landlines and SMS, for literally calling from anywhere to anywhere.

I have to say that I use Skyscanner a lot, and feel it's saved me a tremendous amount of money. The only caveat is that some of the smallest new budget airlines are not included. For accommodation, I have tried both Couchsurfing and Airbnb for the first time this trip. I've actually found Couchsurfing more useful for meeting interesting, colorful people at my destinations than for easily organizing free overnights. I have also been surprised by the amount of people running accommodation operations under the radar via Airbnb, rather than truly private persons renting spare rooms, though I've been satisfied with the service.

Problematic as it may be, I do find myself using Wikitravel precisely because it provides less information rather than more. I like Wikitravel because it gives me a really quick overview of the high points, what to see and do, even for out of the way destinations. I sometimes download the entire page for offline reading, when there is no wifi available. I find both its strongest and weakest points are accommodation. Strong because anyone can add to it, so often gets places under the radar, and weak because it's totally unorganized and un-rated. Because of this deficiency, I find myself often referring to TripAdvisor reviews to double check the lower end accommodations.

Another relatively new technology I'm using a lot is Google Maps, of course frequently for directions, but also particularly for the "Search nearby" capability. I find Google Maps Search nearby capability delightful for discovering new places of all kinds, many perhaps never visited by tourists before. Especially in India directions should be taken with a grain of salt, because places usually seem to be "pinned" imprecisely, so caveat emptor. I find screenshots great for capturing Google Maps directions, easily cut and paste with the "prt sc" key for convenient offline reading.

More than ever before, travel for me is more about people than places. These days I love to visit with friends, old and new, at home or abroad. The reality is that people are now using Facebook more for personal communications than email. Facebook even makes it easier for people traveling in the same regions to meet up along the way. Another reality is that places, particularly low end places, are as likely to have Facebook pages as websites, essentially turning Facebook into its own parallel universe – no other Internet required. In fact, I only use Facebook for people, places/pages and events – and no other bells or whistles, such as innumerable travel apps, etc.

It was Facebook (and my nieces) that finally made me break down and get a small camera (Nikon Coolpix) for posting travel snaps as I go along, often the same day. Though that could be the number one use for phones I'm seeing on the road, not only for taking pictures but also for uploading them in virtual realtime….

04 September 2013

Dissecting the Summarization Process

This is in effect a mid-2013 progress update. As with many of my blog posts, this is as much a status update for me to get a better handle on where I'm at as it is to broadcast my progress.

mendicott.com is a blog reflecting on my journey with the overall project. This blog started seven years ago, in 2006, with my inquiry into The difference between a web page and a blog.... I had then returned from something like five years of world travel to find the digerati fawning over the blogosphere. At first, I failed to see the difference between a blog and a content management system (CMS) for stock standard web pages. Upon closer examination, I began to realize that the real difference lay in the XML syndication of blog feeds into the real-time web.

meta-guide.com is an attempt to blueprint, or tutorialize, the process. My original Meta Guide 1.0 development in ASP attempted to create automated, or robotic, web pages based on XML feeds from the real-time web. Meta Guide 2.0 development was based on similar feed bots, or Twitter bots, in an attempt to automate, or at least semi-automate, the rapid development of large knowledgebases from social media via knowledge silos. Basically, I use knowledge templates to automatically create the knowledge silos, or large knowledgebases. The knowledge templates are based on my own, proprietary "taxonomies", or more precisely faceted classifications, painstakingly developed over many years.

gaiapassage.com aims to be an automated, or semi-automated, summarization of the knowledge aggregated from social media by feed bots via the proprietary faceted classifications, or knowledge templates. Right now, I'm doing a semi-automated summarization process with Gaia Passage, which consists of automated research in the form of knowledge silos being "massaged" in different ways, but ultimately manually writing the summarization in natural language. This is allowing me to analyze and attempt to dissect the processes involved in order to gradually prototype automation. Summarization technologies, and in particular summarization APIs, are still in their infancy. Examples of currently available summarization technologies include automatedinsights.com and narrativescience.com. The overall field is often referred to as automatic summarization.

In the future, the Gaia Passage human readable summarizations will need to be converted into machine readable dialog system knowledgebase format. The dialog system is basically a chatbot, or conversational user interface (CUI) into a specialized database, called a knowledgebase. Most, common chatbot knowledgebases are based on, or compatible with, XML, such as AIML for example. Voice technologies, both output and input, are generally an additional layer on top of the text based dialog system.

The two main bottlenecks I've come up against are what I like to call artificial intelligence middleware, or frameworks, the "glue" to integrate the various processes, as well as adequate dialog system tools, in particular chatbot knowledgebase tools with both "frontend" and "backend" APIs (application programming interface), in other words a dialog system API on the frontend with a backend API into the knowledgebase for dynamic modification. My favorite cloud based "middleware" is Yahoo! Pipes, which is generally referred to as a mashup platform (aka mashup enabler) for feed based data; however, there are severe performance issues with Yahoo! Pipes -- so, I don't really consider it to be a production ready tool. Like Yahoo! Pipes, my ideal visual, cloud based AI middleware could or should be language agnostic -- eliminating the need to decide on a single programming language for a project. I have also looked into scientific computing packages, such as LabVIEW, Mathematica, and MATLAB, for use as potential AI middleware. Additionally, there are a variety of both natural language and intelligent agent frameworks available. Business oriented cloud based integration, including visual cloud based middleware, is often referred to as iPaaS (integration Platform as a Service), integration PaaS or "Integration as a Service".

The recent closure of the previously open Twitter API with OAuth has set my feed bot, or "smart feed", development back by years. Right now, I'm stuck trying to figure out the best way to use the new Twitter OAuth with Yahoo! Pipes, for instance via YQL, if at all. And if that were not enough, the affordable and user-friendly dialog system API, verbotsonline.com, that I was using went out of business. There are a number of dialog system API alternatives, even cloud based dialog systems, but they are neither free nor cheap, especially for significant throughput volumes. Still to do: 1) complete the Gaia Passage summarizations, 2) make Twitter OAuth work, use a commercial third party data source (such as datasift.com, gnip.com or topsy.com), or abandon Twitter as a primary source (for instance concentrate on other social media APIs instead, such as Facebook), 3) continue the search for a new and better dialog system API provider.

Most basically, the Gaia Passage project is a network of robots that will not only monitor social media buzz about both the environment and tourism but also interpret the inter-relations, cause and effects, between environment and tourism -- such as how climate change effects the tourism industry both negatively or positively, or even what effects the weather has on crime trends for a particular destination -- as well as querying these interpreted inter-relations, or "conclusions", via natural language. If this can be accomplished with any degree of satisfaction, either fully automated or semi-automated, then the system could just as easily be applied to any other vertical. Proposals from potential sponsors, investors, or technology partners are welcomed, and may be sent to mendicot [at] yahoo.com.

13 March 2013

A New Website For A New Age: GaiaPassage.com

GaiaPassage.com is subtitled "Marcus L Endicott's favorite tips for green travel around the world".  I'm calling it a deep green, eco-centric travel guide to the whole Earth.  My Gaia Passage project will be a handwritten ecotourism guide to the entire world, based on the circa 250x ccTLD.  The general idea is to write a "white paper" for every country in the world, on environmental and cultural conditions, issues, and who is doing what about them, as well as examining both how they affect tourism and how tourism affects those issues. Anyone could write a lot about something, but the idea here is to provide "snapshots", or "bite sized" summaries, of only the best information and contacts.  The name "Gaia Passage" originally came from my pre-Internet (mid-1980s) travel tips newsletter. The site is a work in progress; so far, I've completed the entire Western Hemisphere:
GaiaPassage.com is handwritten, but based on automated research and automated outline. Primary research is based on data mining 20 years of Green Travel archives. Secondary research is based on multiple years of Meta Guide Twitter bots archives. Significance is based on primary sources in the form of root website domains, and/or secondary sources in the form of Wikipedia entries. In other words, if there is not a root website domain name or a Wikipedia entry then it is unlikely to be included. (However, almost anything may be included in Wikipedia - if properly referenced.) 

I have noticed that many websites of smaller concerns are going down, offline, apparently due to the economic downturn. However, social media such as Twitter and Facebook do present affordable alternatives to owning a root domain website, and I will take these into consideration when appropriate. (In other words, when something is really cool.)  I have also noticed a lot of people using Weebly to make free websites. (Note, GaiaPassage.com currently uses the free Google Sites platform.)

In the early evolution of a website, especially large projects, it's important to first have the "containers" in place as "placeholders", which is no small task in itself. With circa 250x countries and territorial entities, that's a whole year's fulltime work for one man, revising one country per working day. This would mean initial completion by December 2013. Eventually, GaiaPassage.com entries may morph into socialbots, or conversational assistants, containing not only all the knowledge about sustainable tourism gleaned from past Green Travel archives, but also current knowledge resulting from the Meta Guide Twitter bots.

In my previous blog, 250 Conversational Twitter Bots for Travel & Tourism, I detailed my 250x Meta Guide Twitter bots, one for every country and territory in the Internet ccTLD.  Basically, I've spent the past five years working on artificial intelligence and conversational agents - and tweeting about it all the while (links below).  I had been using Twitter extensively as a framework; however, Twitter has become increasingly protectionistic, most dramatically illustrated by the high profile 2012 Twitter-LinkedIn divorce. The Twitter API has become a moving target, which is just too costly for me to keep playing catch up.  In short, I find the "Facebook complex" of Twitter management immensely annoying, and concluded to stop contributing original content; so, my New Year's resolution was to stop tweeting manually at least for all of 2013.  Further, my excellent dialog system API, VerbotsOnline.com, went out of business in 2012.  Any other good dialog system API I found to replace it turned out to be much too expensive.  As a result, all my conversational agents are shut down, at least for 2013.  My hope is that the sector will shake out and/or advance during the year, and better or at least more affordable conversational tools will become available next year.

19 June 2012

250 Conversational Twitter Bots for Travel & Tourism

The reason I haven't updated this blog in almost a year is that I have moved most online development to my Meta-Guide.com website. In the previous two postings, I began testing my content repatriation strategy, in other words aggregating my own content from around the web, which I've continued on the Meta Guide website, in fact concentrating on seeding new webpages from mining the past four years of my own tweets. I have also made a prototype summarizer, which I am now training on my Meta Guide website in order to extract content from it to add on top of the mined tweets when building out new webpages. At the moment, I have three immediate goals. I would like to reach 10,000 tweets, 1,000 Meta Guide webpages, and 100 theses in AI & NLP (from the past 10 years). I only have about another 3,000 tweets to go, so maybe another year, about 300 webpages left to make, and less than 30 more theses to discover.

This past weekend, within view of the spectacular Colorado Rocky Mountains, I succeeded after some struggle in making my 250x Meta Guide Twitter bots conversationally interactive on Twitter. These are 250x manually constructed Twitter bots, one for every country, based on country code top-level domain. That includes one for each of the 193 member states of the United Nations, plus an additional 57 various and sundry territories included in the ccTLD. All of these Meta Guide Twitter bots are powered by my @VagaBot, a single cloud-based Verbot engine from Verbotsonline, using the undocumented API and connected to Twitter via Yahoo! Pipes. Previously they have just been retweeters, aggregating country-specific travel and tourism tweets. The next phase of development will involve marrying the incoming retweets to the outgoing responses in some meaningful way, in other words datamining the incoming retweets and attempting to process them semantically into answers.

You should now be able to @sign tweet any of the Meta Guide Twitter bots with questions. Currently, message turnaround time is running up to 30 minutes, but which is par for Twitter. Among other things replies contain lines from my travel books, see Vagabond Globetrotting 3 & From the Balkans to the Baltics. If you are interested in learning more about me and what I do, I recommend watching both Part 1 & Part 2 of my recent videos on "Open Chatbot Standards for a Modular Chatbot Framework", presented in Philadelphia at Chatbots 3.2: Fifth Colloquium On Conversational Systems. If you need help with socialbots for your social CRM, I am available for consulting; just check my Contact page for details, follow me on Twitter, or connect on LinkedIn, and let's Skype!