Wednesday, July 2, 2014

A leap in artificial intelligence ? Continued - Building the beast.

Building Gaia


After almost 6 months of silence in this blog (due to some other activities) I felt the need to share the actual state of my ideas on the

Artificial intelligence that learns from associated parallel streams of information

Last sunday (June 29th 2014) I started building Gaia. That's the working name I use for this AI.

As I'm a man, I cannot give birth to a living creature called human, but as a transformer of coffee and pizza into bits and bytes I can give birth to something else.

In my tag line in Google+ "Desire a self driving car and other smart companions (male, female and undefined)" one of the "undefined" will be this AI.

I don't like to refer to this AI with the dispassionate "it" so I'll use the term "she" when referring to Gaia.

Inspired by the "World of null-A" by A. E. van Vogt and to be clear that I very well know that this program is not a person I'll use the glyph she (being the non-human she).



Building philosophy and architecture

The most important axiom for building her is that I want to give her as less as possible "hard-wired" instructions on how to do something.
I decided to use a "toolbox" like paradigm.

Typical elements in her toolbox will be:

  • goals
  • algorithms
  • kinds of memory
  • ...

Each of these tools can have discoverable parameters for which she can change the values. A tool has also a set of evaluation algorithms with their own parameters. She can choose and the evaluation algorithm (if there is more then one) and the evaluation parameters. Together they form the evaluation criteria.

She is free to choose whatever combination of goals, algorithms (with their parameters) etcetera she is going to use and evaluate the effectiveness (related to a goal) of it.

The outcome is stored in a special memory (hard coded for this one): the experience center.

Version 0.1 of Gaia's architecture. The components with a red border have been build on 2014-06-30.


The effectiveness of a tool can vary over time. For example a tool might not be useful in her early stage of development but only later on.


Sensor channels

A sensor channel is composed of one or more Sensor Output Consumers. The channel allows the synchronization of the data streams.
A video camera e.g. would be connected to one channel with 2 or 3 consumers: 1 for the video and 1 (mono) or 2 (stereo) for the sound.
A YouTube video might have an extra consumer for the subtitles. Even one for each language when in the future it will be possible to have the subtitles in different languages at the same time (blink, hint, blink to those who are concerned).
A sensor reading digital texts will have only 1 consumer in the associated channel.

The Sensor Output Consumers are initially all identical. None is specific for a particular kind of data.
She has to "train" them to make sense of the data stream. Using the mechanism described above.
To improve the overall effectiveness it is recommended (but not necessary) to keep the data flow between a sensor and an associated consumer homogenous (e.g. only streaming video or only streaming sound).

The reason why I opted for this solution is that I can define what kinds of data streams can be made available now. But I don't know what will be available tomorrow. A "smell" stream perhaps?

The Sensor Output Consumer tool should be flexible enough to cope with new kinds of data streams.


The data consumed

To avoid what is called "over fitting" in traditional AI, Gaia will see a particular stream of data only once. Depending on the tool used she will retain only a very small fraction of the first streams but this fraction should increase more or less rapidly afterwards.


What has been done so far

To facilitate the architecture implementation to mature over time, (yes, this first implementation will not be the last), I started with an easy data stream: text.

The data sources I use are free eBooks from the Gutenberg project. And my compliments for all the volunteers making the books available.
The books are in different languages. For now in English, French, German, Spanish and Latin.
I selected these languages because it allows me to identify what I would like to see what she has done with it and compare that with what she has really done.

And there were surprises.

After the first few testing rounds, what I saw in her memory was far away from what I expected to see.

The books from the Gutenberg project have all, at the end, a license section. These sections are almost identical for all books. She considered these as more important then the rest and it overshadowed everything else.

Note: the term considered should not be taken literally. She is far from being able to consider whatever.

This is typically a case of over fitting.
To cope with this I choose to keep only the text of the books and strip of the headers and tails.

Her implicit goal for now could be described as:

Fill my memory with frequent patterns of bytes

The Sensor Output Consumer tool she is using has 6 configurable numeric parameters and 1 boolean for 1 algorithm.
For the evaluation she has 3 extra parameters and 4 algorithms to choose from.

The numeric parameters are not constraint in any way. Providing a negative value for something that should be used as a counter will terminate the experience immediately for instance.
And the evaluation for each of the evaluation algorithms will be bad (with regards to the goal).

Fiddling around with her options on a 40 M data stream for less then a minute produces the JSON memory dump add the end of this post.

The existing "Pattern splitting" tool has not been used yet.
This tool splits patterns in memory into 2 other patterns if each of these exist. The original pattern will be removed and the 2 resulting patterns will have their count (firing rate) increased.
Doing so with isolated patterns will imply the lost of precious information. Because the patterns are sequentially related (sequence in time or space as you like) and this information will now be lost.
And I'm not yet fixed if I should make the evaluation criteria for this stage of her development. The availability of evaluation criteria allows her to use the tool.



Avoiding misunderstanding

In this post, as it will be the case in the next ones, I describe Gaia, the new AI, as if it were a person.
This is a personal choice, that helps me thinking about what and how the AI should do things.
Not thinking about her as a bunch of algorithms, which in reality it is of course.

To be explicit:

Gaia doesn't have any perception yet on what she is handling. She only tries to find frequent patterns in a stream.
Even her perception of the term frequent is reduced to a function result (the count() function).

Her actual data memory is her infant memory. There are only isolated patterns without any associations / links between them.
I will soon add a new kind of memory that allows patterns to be connected in various ways.


Roadmap

There is a multitude of next steps on Gaia's development agenda. To mention a few:

  • provide video streams
  • provide sound streams
  • create her next generation memory
  • develop other algorithms
  • improve existing algorithms
  • improve and extend the architecture framework
  • make the code open source
  • ....


What and when things will be done depends on a lot of factors, for which some of them are unfortunately out of my sphere of influence.

If you're interested stay tuned. I'll keep you informed in this blog about the progress on Gaia.

In the mean time relax and think about the implications (benefits and risks).


Financial support


If you like to support the development of this new kind of AI you can donate Bitcoins (or fragments of it) at

1HadqhKD5EdCDpTorgxP9P8akKqycEUkwk


When Gaia will be more mature and she will be able to interact with the environment she might also need to spent money. She is not there yet but Bitcoin donations for her can already been done at:

1PyyRd6k5kSuuErAnJNpvdoNCW5ZPgD9fb





I'm excited about this piece of software.

And you?


Ronald Poell
2014-07-02


First chapter Previous chapter


Edit 2014-07-02: corrected a forgotten she . When the glyph cannot be written use null-she.


Below you'll find her memory dump after running from scratch (empty memory and no previous experiences) on a 40 M stream of text.

In some places you will see the well known � character.
As she doesn't know what she is looking at (in this case a UTF-8 stream) she doesn't know yet about code points in UTF-8. And when a code point is not included in a multi byte character (the UTF-8 character is not well formed) the parser in e.g. a web browser will display the � character.

It will be interesting to see how long it will take her to get rid of them.


{
    "shortest": 2,
    "longest": 10,
    "patterns": [
        {
            "pattern": "0D0A (\r\n)",
            "count": 6725,
            "length": 2
        },
        {
            "pattern": "6520 (e )",
            "count": 6556,
            "length": 2
        },
        {
            "pattern": "202020202020 (      )",
            "count": 4823,
            "length": 6
        },
        {
            "pattern": "20202020202020 (       )",
            "count": 3842,
            "length": 7
        },
        {
            "pattern": "7320 (s )",
            "count": 3196,
            "length": 2
        },
        {
            "pattern": "2C20 (, )",
            "count": 3137,
            "length": 2
        },
        {
            "pattern": "206F6620746865 ( of the)",
            "count": 2583,
            "length": 7
        },
        {
            "pattern": "206F66207468 ( of th)",
            "count": 2483,
            "length": 6
        },
        {
            "pattern": "6F6620746865 (of the)",
            "count": 2475,
            "length": 6
        },
        {
            "pattern": "C3A97461697420 (était )",
            "count": 2279,
            "length": 7
        },
        {
            "pattern": "662074686520 (f the )",
            "count": 2122,
            "length": 6
        },
        {
            "pattern": "6F662074686520 (of the )",
            "count": 2112,
            "length": 7
        },
        {
            "pattern": "2E0D0A0D0A2D2D (.\r\n\r\n--)",
            "count": 2036,
            "length": 7
        },
        {
            "pattern": "206465206C6120 ( de la )",
            "count": 2017,
            "length": 7
        },
        {
            "pattern": "6E20 (n )",
            "count": 1990,
            "length": 2
        },
        {
            "pattern": "20202020202020202020 (          )",
            "count": 1836,
            "length": 10
        },
        {
            "pattern": "6E2074686520 (n the )",
            "count": 1820,
            "length": 6
        },
        {
            "pattern": "207468617420 ( that )",
            "count": 1773,
            "length": 6
        },
        {
            "pattern": "2C20616E6420 (, and )",
            "count": 1748,
            "length": 6
        },
        {
            "pattern": "6C20 (l )",
            "count": 1684,
            "length": 2
        },
        {
            "pattern": "2064616E73206C ( dans l)",
            "count": 1597,
            "length": 7
        },
        {
            "pattern": "656E74 (ent)",
            "count": 1573,
            "length": 3
        },
        {
            "pattern": "2E20 (. )",
            "count": 1544,
            "length": 2
        },
        {
            "pattern": "2020 (  )",
            "count": 1465,
            "length": 2
        },
        {
            "pattern": "20776869636820 ( which )",
            "count": 1369,
            "length": 7
        },
        {
            "pattern": "207468652073 ( the s)",
            "count": 1282,
            "length": 6
        },
        {
            "pattern": "20696E207468 ( in th)",
            "count": 1221,
            "length": 6
        },
        {
            "pattern": "696E20746865 (in the)",
            "count": 1200,
            "length": 6
        },
        {
            "pattern": "20746865 ( the)",
            "count": 1136,
            "length": 4
        },
        {
            "pattern": "20636574746520 ( cette )",
            "count": 1131,
            "length": 7
        },
        {
            "pattern": "6F6E (on)",
            "count": 1114,
            "length": 2
        },
        {
            "pattern": "207468650D0A ( the\r\n)",
            "count": 1092,
            "length": 6
        },
        {
            "pattern": "202020202020202020 (         )",
            "count": 1059,
            "length": 9
        },
        {
            "pattern": "207768696368 ( which)",
            "count": 1028,
            "length": 6
        },
        {
            "pattern": "20C3A974616974 ( était)",
            "count": 1021,
            "length": 7
        },
        {
            "pattern": "207769746820 ( with )",
            "count": 1018,
            "length": 6
        },
        {
            "pattern": "776869636820 (which )",
            "count": 1001,
            "length": 6
        },
        {
            "pattern": "20696E20746865 ( in the)",
            "count": 992,
            "length": 7
        },
        {
            "pattern": "616E (an)",
            "count": 951,
            "length": 2
        },
        {
            "pattern": "64616E73206C65 (dans le)",
            "count": 948,
            "length": 7
        },
        {
            "pattern": "20636F6D6D6520 ( comme )",
            "count": 943,
            "length": 7
        },
        {
            "pattern": "74686520 (the )",
            "count": 936,
            "length": 4
        },
        {
            "pattern": "0D0A0D0A2D2D20 (\r\n\r\n-- )",
            "count": 926,
            "length": 7
        },
        {
            "pattern": "656E7420C3A020 (ent à )",
            "count": 924,
            "length": 7
        },
        {
            "pattern": "2020202020202020 (        )",
            "count": 921,
            "length": 8
        },
        {
            "pattern": "696E2074686520 (in the )",
            "count": 917,
            "length": 7
        },
        {
            "pattern": "7261 (ra)",
            "count": 876,
            "length": 2
        },
        {
            "pattern": "7175656C717565 (quelque)",
            "count": 847,
            "length": 7
        },
        {
            "pattern": "206861766520 ( have )",
            "count": 844,
            "length": 6
        },
        {
            "pattern": "742074686520 (t the )",
            "count": 835,
            "length": 6
        },
        {
            "pattern": "20746F20746865 ( to the)",
            "count": 831,
            "length": 7
        },
        {
            "pattern": "20616E642074 ( and t)",
            "count": 828,
            "length": 6
        },
        {
            "pattern": "642074686520 (d the )",
            "count": 802,
            "length": 6
        },
        {
            "pattern": "6520C3A9746169 (e étai)",
            "count": 795,
            "length": 7
        },
        {
            "pattern": "20617661697420 ( avait )",
            "count": 795,
            "length": 7
        },
        {
            "pattern": "61697420706173 (ait pas)",
            "count": 789,
            "length": 7
        },
        {
            "pattern": "746F2074686520 (to the )",
            "count": 785,
            "length": 7
        },
        {
            "pattern": "20746F207468 ( to th)",
            "count": 775,
            "length": 6
        },
        {
            "pattern": "7265 (re)",
            "count": 775,
            "length": 2
        },
        {
            "pattern": "74686174207468 (that th)",
            "count": 771,
            "length": 7
        },
        {
            "pattern": "746F20746865 (to the)",
            "count": 763,
            "length": 6
        },
        {
            "pattern": "74696F6E206F66 (tion of)",
            "count": 762,
            "length": 7
        },
        {
            "pattern": "2066726F6D20 ( from )",
            "count": 761,
            "length": 6
        },
        {
            "pattern": "20717565206C65 ( que le)",
            "count": 753,
            "length": 7
        },
        {
            "pattern": "6573 (es)",
            "count": 753,
            "length": 2
        },
        {
            "pattern": "617420746865 (at the)",
            "count": 751,
            "length": 6
        },
        {
            "pattern": "652073 (e s)",
            "count": 751,
            "length": 3
        },
        {
            "pattern": "207175656C7175 ( quelqu)",
            "count": 740,
            "length": 7
        },
        {
            "pattern": "2072C3A9706F6E ( répon)",
            "count": 739,
            "length": 7
        },
        {
            "pattern": "6520706F757220 (e pour )",
            "count": 728,
            "length": 7
        },
        {
            "pattern": "20616E64207468 ( and th)",
            "count": 721,
            "length": 7
        },
        {
            "pattern": "0D0A2020202020 (\r\n     )",
            "count": 718,
            "length": 7
        },
        {
            "pattern": "206F6620 ( of )",
            "count": 713,
            "length": 4
        },
        {
            "pattern": "27C3A974616974 ('était)",
            "count": 713,
            "length": 7
        },
        {
            "pattern": "20717527696C20 ( qu'il )",
            "count": 712,
            "length": 7
        },
        {
            "pattern": "696F6E206F6620 (ion of )",
            "count": 711,
            "length": 7
        },
        {
            "pattern": "616E64207468 (and th)",
            "count": 705,
            "length": 6
        },
        {
            "pattern": "2020202020 (     )",
            "count": 705,
            "length": 5
        },
        {
            "pattern": "6F2074686520 (o the )",
            "count": 702,
            "length": 6
        },
        {
            "pattern": "6E74206C657320 (nt les )",
            "count": 700,
            "length": 7
        },
        {
            "pattern": "20C3A0206C6120 ( à la )",
            "count": 698,
            "length": 7
        },
        {
            "pattern": "207468652063 ( the c)",
            "count": 697,
            "length": 6
        },
        {
            "pattern": "696F6E206F66 (ion of)",
            "count": 696,
            "length": 6
        },
        {
            "pattern": "7269 (ri)",
            "count": 694,
            "length": 2
        },
        {
            "pattern": "20746861742074 ( that t)",
            "count": 694,
            "length": 7
        },
        {
            "pattern": "657220 (er )",
            "count": 692,
            "length": 3
        },
        {
            "pattern": "61742074686520 (at the )",
            "count": 692,
            "length": 7
        },
        {
            "pattern": "72C3A9706F6E64 (répond)",
            "count": 690,
            "length": 7
        },
        {
            "pattern": "65206F66207468 (e of th)",
            "count": 681,
            "length": 7
        },
        {
            "pattern": "6E6420746865 (nd the)",
            "count": 675,
            "length": 6
        },
        {
            "pattern": "6172 (ar)",
            "count": 674,
            "length": 2
        },
        {
            "pattern": "68617420746865 (hat the)",
            "count": 669,
            "length": 7
        },
        {
            "pattern": "C3A872656E7420 (èrent )",
            "count": 664,
            "length": 7
        },
        {
            "pattern": "6F6E206F6620 (on of )",
            "count": 661,
            "length": 6
        },
        {
            "pattern": "0D0A74686520 (\r\nthe )",
            "count": 657,
            "length": 6
        },
        {
            "pattern": "20737572206C65 ( sur le)",
            "count": 646,
            "length": 7
        },
        {
            "pattern": "EFBFBD706F6E646974 (�pondit)",
            "count": 642,
            "length": 9
        },
        {
            "pattern": "73206465206C61 (s de la)",
            "count": 642,
            "length": 7
        },
        {
            "pattern": "65206465206C61 (e de la)",
            "count": 638,
            "length": 7
        },
        {
            "pattern": "2074686520 ( the )",
            "count": 638,
            "length": 5
        },
        {
            "pattern": "74696F6E206F (tion o)",
            "count": 636,
            "length": 6
        },
        {
            "pattern": "746169656E7420 (taient )",
            "count": 636,
            "length": 7
        },
        {
            "pattern": "696E (in)",
            "count": 633,
            "length": 2
        },
        {
            "pattern": "20C3A974C3A920 ( été )",
            "count": 629,
            "length": 7
        },
        {
            "pattern": "207468652070 ( the p)",
            "count": 627,
            "length": 6
        },
        {
            "pattern": "C3A9706F6E6469 (épondi)",
            "count": 606,
            "length": 7
        },
        {
            "pattern": "742064616E7320 (t dans )",
            "count": 605,
            "length": 7
        },
        {
            "pattern": "652061 (e a)",
            "count": 602,
            "length": 3
        },
        {
            "pattern": "20746865206D ( the m)",
            "count": 602,
            "length": 6
        },
        {
            "pattern": "73206F662074 (s of t)",
            "count": 598,
            "length": 6
        },
        {
            "pattern": "656E7420646520 (ent de )",
            "count": 593,
            "length": 7
        },
        {
            "pattern": "726F (ro)",
            "count": 578,
            "length": 2
        },
        {
            "pattern": "207468652066 ( the f)",
            "count": 571,
            "length": 6
        },
        {
            "pattern": "6174696F6E20 (ation )",
            "count": 570,
            "length": 6
        },
        {
            "pattern": "202020 (   )",
            "count": 564,
            "length": 3
        },
        {
            "pattern": "207468697320 ( this )",
            "count": 557,
            "length": 6
        },
        {
            "pattern": "686174207468 (hat th)",
            "count": 545,
            "length": 6
        },
        {
            "pattern": "6F6E20746865 (on the)",
            "count": 532,
            "length": 6
        },
        {
            "pattern": "652074686520 (e the )",
            "count": 531,
            "length": 6
        },
        {
            "pattern": "65206F662074 (e of t)",
            "count": 530,
            "length": 6
        },
        {
            "pattern": "0D0A0D0A2020 (\r\n\r\n  )",
            "count": 530,
            "length": 6
        },
        {
            "pattern": "732074686520 (s the )",
            "count": 529,
            "length": 6
        },
        {
            "pattern": "2C2074686520 (, the )",
            "count": 528,
            "length": 6
        },
        {
            "pattern": "746861742074 (that t)",
            "count": 527,
            "length": 6
        },
        {
            "pattern": "6572 (er)",
            "count": 524,
            "length": 2
        },
        {
            "pattern": "696E67207468 (ing th)",
            "count": 509,
            "length": 6
        },
        {
            "pattern": "0D0A20202020 (\r\n    )",
            "count": 507,
            "length": 6
        },
        {
            "pattern": "656420746F20 (ed to )",
            "count": 494,
            "length": 6
        },
        {
            "pattern": "7320616E6420 (s and )",
            "count": 494,
            "length": 6
        },
        {
            "pattern": "207468652065 ( the e)",
            "count": 484,
            "length": 6
        },
        {
            "pattern": "207468657920 ( they )",
            "count": 483,
            "length": 6
        },
        {
            "pattern": "7C0D0A20202020207C (|\r\n     |)",
            "count": 449,
            "length": 9
        },
        {
            "pattern": "616C (al)",
            "count": 401,
            "length": 2
        },
        {
            "pattern": "6F6D (om)",
            "count": 390,
            "length": 2
        },
        {
            "pattern": "20746F ( to)",
            "count": 384,
            "length": 3
        },
        {
            "pattern": "207468 ( th)",
            "count": 379,
            "length": 3
        },
        {
            "pattern": "7274 (rt)",
            "count": 347,
            "length": 2
        },
        {
            "pattern": "20696E20 ( in )",
            "count": 334,
            "length": 4
        },
        {
            "pattern": "616E6420 (and )",
            "count": 330,
            "length": 4
        },
        {
            "pattern": "204D ( M)",
            "count": 319,
            "length": 2
        },
        {
            "pattern": "6E61 (na)",
            "count": 312,
            "length": 2
        },
        {
            "pattern": "66207468 (f th)",
            "count": 311,
            "length": 4
        },
        {
            "pattern": "2C2061 (, a)",
            "count": 311,
            "length": 3
        },
        {
            "pattern": "20616E64 ( and)",
            "count": 307,
            "length": 4
        },
        {
            "pattern": "6E6420 (nd )",
            "count": 305,
            "length": 3
        },
        {
            "pattern": "7469 (ti)",
            "count": 300,
            "length": 2
        },
        {
            "pattern": "20636F ( co)",
            "count": 293,
            "length": 3
        },
        {
            "pattern": "6F662074 (of t)",
            "count": 289,
            "length": 4
        },
        {
            "pattern": "20746F20 ( to )",
            "count": 285,
            "length": 4
        },
        {
            "pattern": "746865 (the)",
            "count": 281,
            "length": 3
        },
        {
            "pattern": "696E6720 (ing )",
            "count": 277,
            "length": 4
        },
        {
            "pattern": "202022202020207C20 (  \"    | )",
            "count": 277,
            "length": 9
        },
        {
            "pattern": "204E ( N)",
            "count": 274,
            "length": 2
        },
        {
            "pattern": "206120 ( a )",
            "count": 274,
            "length": 3
        },
        {
            "pattern": "206F ( o)",
            "count": 273,
            "length": 2
        },
        {
            "pattern": "6D20 (m )",
            "count": 267,
            "length": 2
        },
        {
            "pattern": "74696F6E (tion)",
            "count": 264,
            "length": 4
        },
        {
            "pattern": "6E67 (ng)",
            "count": 262,
            "length": 2
        },
        {
            "pattern": "6F72 (or)",
            "count": 259,
            "length": 2
        },
        {
            "pattern": "6E207468 (n th)",
            "count": 258,
            "length": 4
        },
        {
            "pattern": "686520 (he )",
            "count": 249,
            "length": 3
        },
        {
            "pattern": "2047 ( G)",
            "count": 232,
            "length": 2
        },
        {
            "pattern": "2069 ( i)",
            "count": 231,
            "length": 2
        },
        {
            "pattern": "6C6F6265 (lobe)",
            "count": 218,
            "length": 4
        },
        {
            "pattern": "6620746865 (f the)",
            "count": 200,
            "length": 5
        },
        {
            "pattern": "20616E642074686520 ( and the )",
            "count": 200,
            "length": 9
        },
        {
            "pattern": "617420 (at )",
            "count": 198,
            "length": 3
        },
        {
            "pattern": "6865 (he)",
            "count": 197,
            "length": 2
        },
        {
            "pattern": "20616E6420 ( and )",
            "count": 196,
            "length": 5
        },
        {
            "pattern": "65206F662074686520 (e of the )",
            "count": 195,
            "length": 9
        },
        {
            "pattern": "656420 (ed )",
            "count": 187,
            "length": 3
        },
        {
            "pattern": "20676C6F ( glo)",
            "count": 186,
            "length": 4
        },
        {
            "pattern": "746865206561727468 (the earth)",
            "count": 185,
            "length": 9
        },
        {
            "pattern": "206F662074 ( of t)",
            "count": 182,
            "length": 5
        },
        {
            "pattern": "206F66207468652073 ( of the s)",
            "count": 179,
            "length": 9
        },
        {
            "pattern": "2072C3A9706F6E64 ( répond)",
            "count": 176,
            "length": 8
        },
        {
            "pattern": "6F66207468 (of th)",
            "count": 176,
            "length": 5
        },
        {
            "pattern": "2D2D2D2D2D2D2D2D2D2D (----------)",
            "count": 175,
            "length": 10
        },
        {
            "pattern": "696F6E20 (ion )",
            "count": 173,
            "length": 4
        },
        {
            "pattern": "65206F66 (e of)",
            "count": 172,
            "length": 4
        },
        {
            "pattern": "0D0A0D0A (\r\n\r\n)",
            "count": 170,
            "length": 4
        },
        {
            "pattern": "616E64 (and)",
            "count": 170,
            "length": 3
        },
        {
            "pattern": "6C6F (lo)",
            "count": 169,
            "length": 2
        },
        {
            "pattern": "68697320 (his )",
            "count": 167,
            "length": 4
        },
        {
            "pattern": "68696368 (hich)",
            "count": 164,
            "length": 4
        },
        {
            "pattern": "73206F662074686520 (s of the )",
            "count": 163,
            "length": 9
        },
        {
            "pattern": "6F6620 (of )",
            "count": 162,
            "length": 3
        },
        {
            "pattern": "6620 (f )",
            "count": 160,
            "length": 2
        },
        {
            "pattern": "676C6F62 (glob)",
            "count": 160,
            "length": 4
        },
        {
            "pattern": "2E0D0A0D0A2D2D20 (.\r\n\r\n-- )",
            "count": 157,
            "length": 8
        },
        {
            "pattern": "66726F6D2074686520 (from the )",
            "count": 157,
            "length": 9
        },
        {
            "pattern": "74686174 (that)",
            "count": 156,
            "length": 4
        },
        {
            "pattern": "206F66 ( of)",
            "count": 153,
            "length": 3
        },
        {
            "pattern": "20697320 ( is )",
            "count": 153,
            "length": 4
        },
        {
            "pattern": "746F20 (to )",
            "count": 153,
            "length": 3
        },
        {
            "pattern": "2B2D2D2D2D2D2D2D2D (+--------)",
            "count": 152,
            "length": 9
        },
        {
            "pattern": "68617420 (hat )",
            "count": 150,
            "length": 4
        },
        {
            "pattern": "2031 ( 1)",
            "count": 149,
            "length": 2
        },
        {
            "pattern": "20E2809C ( “)",
            "count": 147,
            "length": 4
        },
        {
            "pattern": "74656420 (ted )",
            "count": 147,
            "length": 4
        },
        {
            "pattern": "6E20746865 (n the)",
            "count": 146,
            "length": 5
        },
        {
            "pattern": "77697468 (with)",
            "count": 146,
            "length": 4
        },
        {
            "pattern": "20746861 ( tha)",
            "count": 143,
            "length": 4
        },
        {
            "pattern": "6967 (ig)",
            "count": 143,
            "length": 2
        },
        {
            "pattern": "73206F66 (s of)",
            "count": 143,
            "length": 4
        },
        {
            "pattern": "6265 (be)",
            "count": 142,
            "length": 2
        },
        {
            "pattern": "20776869 ( whi)",
            "count": 141,
            "length": 4
        },
        {
            "pattern": "652074 (e t)",
            "count": 139,
            "length": 3
        },
        {
            "pattern": "746861742074686520 (that the )",
            "count": 139,
            "length": 9
        },
        {
            "pattern": "2D2D2D2D2D2D2D2D (--------)",
            "count": 137,
            "length": 8
        },
        {
            "pattern": "72C3A9706F6E6469 (répondi)",
            "count": 136,
            "length": 8
        },
        {
            "pattern": "77686963 (whic)",
            "count": 136,
            "length": 4
        },
        {
            "pattern": "2C20616E (, an)",
            "count": 136,
            "length": 4
        },
        {
            "pattern": "73206F6620 (s of )",
            "count": 132,
            "length": 5
        },
        {
            "pattern": "20676C6F62 ( glob)",
            "count": 129,
            "length": 5
        },
        {
            "pattern": "207468617420746865 ( that the)",
            "count": 128,
            "length": 9
        },
        {
            "pattern": "6E6720 (ng )",
            "count": 128,
            "length": 3
        },
        {
            "pattern": "206F66207468650D0A ( of the\r\n)",
            "count": 126,
            "length": 9
        },
        {
            "pattern": "20696E2074 ( in t)",
            "count": 124,
            "length": 5
        },
        {
            "pattern": "4BC3A9726162616E (Kéraban)",
            "count": 124,
            "length": 8
        },
        {
            "pattern": "696E2074 (in t)",
            "count": 124,
            "length": 4
        },
        {
            "pattern": "206861 ( ha)",
            "count": 123,
            "length": 3
        },
        {
            "pattern": "74206F662074686520 (t of the )",
            "count": 122,
            "length": 9
        },
        {
            "pattern": "27C3A97461697420 ('était )",
            "count": 120,
            "length": 8
        },
        {
            "pattern": "6174696F (atio)",
            "count": 120,
            "length": 4
        },
        {
            "pattern": "EFBFBD706F6E64697420 (�pondit )",
            "count": 120,
            "length": 10
        },
        {
            "pattern": "636170697461696E (capitain)",
            "count": 119,
            "length": 8
        },
        {
            "pattern": "74686572 (ther)",
            "count": 119,
            "length": 4
        },
        {
            "pattern": "20696E ( in)",
            "count": 119,
            "length": 3
        },
        {
            "pattern": "686520656172746820 (he earth )",
            "count": 117,
            "length": 9
        },
        {
            "pattern": "68652073 (he s)",
            "count": 117,
            "length": 4
        },
        {
            "pattern": "757374726174696F6E (ustration)",
            "count": 116,
            "length": 9
        },
        {
            "pattern": "20C3A97461697420 ( était )",
            "count": 116,
            "length": 8
        },
        {
            "pattern": "65726520 (ere )",
            "count": 116,
            "length": 4
        },
        {
            "pattern": "E2809D20 (” )",
            "count": 115,
            "length": 4
        },
        {
            "pattern": "676C6F6265 (globe)",
            "count": 115,
            "length": 5
        },
        {
            "pattern": "C3A9706F6E646974 (épondit)",
            "count": 114,
            "length": 8
        },
        {
            "pattern": "65206F6620 (e of )",
            "count": 113,
            "length": 5
        },
        {
            "pattern": "20617320 ( as )",
            "count": 113,
            "length": 4
        },
        {
            "pattern": "206D61 ( ma)",
            "count": 112,
            "length": 3
        },
        {
            "pattern": "68650D0A (he\r\n)",
            "count": 112,
            "length": 4
        },
        {
            "pattern": "6170697461696E65 (apitaine)",
            "count": 112,
            "length": 8
        },
        {
            "pattern": "7468650D (the\r)",
            "count": 110,
            "length": 4
        },
        {
            "pattern": "69636820 (ich )",
            "count": 109,
            "length": 4
        },
        {
            "pattern": "696E207468 (in th)",
            "count": 108,
            "length": 5
        },
        {
            "pattern": "68657265 (here)",
            "count": 108,
            "length": 4
        },
        {
            "pattern": "205F ( _)",
            "count": 107,
            "length": 2
        },
        {
            "pattern": "636820 (ch )",
            "count": 106,
            "length": 3
        },
        {
            "pattern": "73207468 (s th)",
            "count": 106,
            "length": 4
        },
        {
            "pattern": "207175656C717565 ( quelque)",
            "count": 106,
            "length": 8
        },
        {
            "pattern": "776974682074686520 (with the )",
            "count": 104,
            "length": 9
        },
        {
            "pattern": "20686973 ( his)",
            "count": 104,
            "length": 4
        },
        {
            "pattern": "207768696368207468 ( which th)",
            "count": 104,
            "length": 9
        },
        {
            "pattern": "2064616E73206C65 ( dans le)",
            "count": 103,
            "length": 8
        },
        {
            "pattern": "2C2072C3A9706F6E (, répon)",
            "count": 103,
            "length": 8
        },
        {
            "pattern": "5F20 (_ )",
            "count": 103,
            "length": 2
        },
        {
            "pattern": "746F2074 (to t)",
            "count": 101,
            "length": 4
        },
        {
            "pattern": "204BC3A972616261 ( Kéraba)",
            "count": 101,
            "length": 8
        },
        {
            "pattern": "20636F6E ( con)",
            "count": 100,
            "length": 4
        },
        {
            "pattern": "6E206F662074686520 (n of the )",
            "count": 99,
            "length": 9
        },
        {
            "pattern": "6F207468 (o th)",
            "count": 99,
            "length": 4
        },
        {
            "pattern": "696F6E (ion)",
            "count": 98,
            "length": 3
        },
        {
            "pattern": "20776173 ( was)",
            "count": 98,
            "length": 4
        },
        {
            "pattern": "2063617069746169 ( capitai)",
            "count": 96,
            "length": 8
        },
        {
            "pattern": "6420696E (d in)",
            "count": 96,
            "length": 4
        },
        {
            "pattern": "6F6620746865207375 (of the su)",
            "count": 94,
            "length": 9
        },
        {
            "pattern": "64207468 (d th)",
            "count": 94,
            "length": 4
        },
        {
            "pattern": "6F62 (ob)",
            "count": 93,
            "length": 2
        },
        {
            "pattern": "6569676E65757220 (eigneur )",
            "count": 92,
            "length": 8
        },
        {
            "pattern": "20726174696F6E65 ( ratione)",
            "count": 91,
            "length": 8
        },
        {
            "pattern": "2020202020207C0D0A (      |\r\n)",
            "count": 90,
            "length": 9
        },
        {
            "pattern": "6520706F75766169 (e pouvai)",
            "count": 89,
            "length": 8
        },
        {
            "pattern": "4E617574696C7573 (Nautilus)",
            "count": 88,
            "length": 8
        },
        {
            "pattern": "76656C6F63697461 (velocita)",
            "count": 88,
            "length": 8
        },
        {
            "pattern": "2C20616E64 (, and)",
            "count": 87,
            "length": 5
        },
        {
            "pattern": "7C2020202020202020 (|        )",
            "count": 87,
            "length": 9
        },
        {
            "pattern": "616E204D69747465 (an Mitte)",
            "count": 87,
            "length": 8
        },
        {
            "pattern": "2C20616E6420746865 (, and the)",
            "count": 84,
            "length": 9
        },
        {
            "pattern": "6F6E736965757220 (onsieur )",
            "count": 84,
            "length": 8
        },
        {
            "pattern": "70697461696E6520 (pitaine )",
            "count": 83,
            "length": 8
        },
        {
            "pattern": "20616E ( an)",
            "count": 83,
            "length": 3
        },
        {
            "pattern": "696E67 (ing)",
            "count": 82,
            "length": 3
        },
        {
            "pattern": "6420696E2074686520 (d in the )",
            "count": 82,
            "length": 9
        },
        {
            "pattern": "74696F6E20 (tion )",
            "count": 82,
            "length": 5
        },
        {
            "pattern": "20617374726F6E6F6D ( astronom)",
            "count": 82,
            "length": 9
        },
        {
            "pattern": "6869636820 (hich )",
            "count": 82,
            "length": 5
        },
        {
            "pattern": "617574696C75735F (autilus_)",
            "count": 81,
            "length": 8
        },
        {
            "pattern": "6174696F6E (ation)",
            "count": 77,
            "length": 5
        },
        {
            "pattern": "6E204D697474656E (n Mitten)",
            "count": 76,
            "length": 8
        },
        {
            "pattern": "6520636170697461 (e capita)",
            "count": 75,
            "length": 8
        },
        {
            "pattern": "7768696368 (which)",
            "count": 75,
            "length": 5
        },
        {
            "pattern": "5F4E617574696C75 (_Nautilu)",
            "count": 74,
            "length": 8
        },
        {
            "pattern": "207768 ( wh)",
            "count": 74,
            "length": 3
        },
        {
            "pattern": "746F207468 (to th)",
            "count": 73,
            "length": 5
        },
        {
            "pattern": "56616E204D697474 (Van Mitt)",
            "count": 73,
            "length": 8
        },
        {
            "pattern": "74696F (tio)",
            "count": 73,
            "length": 3
        },
        {
            "pattern": "73206465206C6120 (s de la )",
            "count": 73,
            "length": 8
        },
        {
            "pattern": "6F6E20 (on )",
            "count": 72,
            "length": 3
        },
        {
            "pattern": "696E20 (in )",
            "count": 72,
            "length": 3
        },
        {
            "pattern": "6174612072617469 (ata rati)",
            "count": 72,
            "length": 8
        },
        {
            "pattern": "0D0A0D0A20202020 (\r\n\r\n    )",
            "count": 72,
            "length": 8
        },
        {
            "pattern": "6D6F6E7369657572 (monsieur)",
            "count": 72,
            "length": 8
        },
        {
            "pattern": "207468650D ( the\r)",
            "count": 72,
            "length": 5
        },
        {
            "pattern": "6E2074 (n t)",
            "count": 71,
            "length": 3
        },
        {
            "pattern": "7468652073 (the s)",
            "count": 71,
            "length": 5
        },
        {
            "pattern": "64697374616E7469 (distanti)",
            "count": 71,
            "length": 8
        },
        {
            "pattern": "736569676E657572 (seigneur)",
            "count": 71,
            "length": 8
        },
        {
            "pattern": "657320 (es )",
            "count": 70,
            "length": 3
        },
        {
            "pattern": "205F4E617574696C ( _Nautil)",
            "count": 70,
            "length": 8
        },
        {
            "pattern": "6769 (gi)",
            "count": 70,
            "length": 2
        },
        {
            "pattern": "6F706F7274696F6E (oportion)",
            "count": 70,
            "length": 8
        },
        {
            "pattern": "726174696F6E6520 (ratione )",
            "count": 69,
            "length": 8
        },
        {
            "pattern": "70726F706F727469 (proporti)",
            "count": 69,
            "length": 8
        },
        {
            "pattern": "652E0D0A0D0A2D2D (e.\r\n\r\n--)",
            "count": 68,
            "length": 8
        },
        {
            "pattern": "7175656C71756573 (quelques)",
            "count": 68,
            "length": 8
        },
        {
            "pattern": "6C6F626520 (lobe )",
            "count": 67,
            "length": 5
        },
        {
            "pattern": "20C3A0206C612073 ( à la s)",
            "count": 67,
            "length": 8
        },
        {
            "pattern": "65206465206C6120 (e de la )",
            "count": 67,
            "length": 8
        },
        {
            "pattern": "2066726F6D20746865 ( from the)",
            "count": 67,
            "length": 9
        },
        {
            "pattern": "74656C6C6174696F6E (tellation)",
            "count": 67,
            "length": 9
        },
        {
            "pattern": "2077686963 ( whic)",
            "count": 66,
            "length": 5
        },
        {
            "pattern": "2079656172 ( year)",
            "count": 66,
            "length": 5
        },
        {
            "pattern": "657270656E646963 (erpendic)",
            "count": 66,
            "length": 8
        },
        {
            "pattern": "2074686174 ( that)",
            "count": 66,
            "length": 5
        },
        {
            "pattern": "2074686973 ( this)",
            "count": 66,
            "length": 5
        },
        {
            "pattern": "7468650D0A (the\r\n)",
            "count": 66,
            "length": 5
        },
        {
            "pattern": "726F706F7274696F (roportio)",
            "count": 65,
            "length": 8
        },
        {
            "pattern": "737472 (str)",
            "count": 65,
            "length": 3
        },
        {
            "pattern": "2077697468 ( with)",
            "count": 65,
            "length": 5
        },
        {
            "pattern": "662074 (f t)",
            "count": 65,
            "length": 3
        },
        {
            "pattern": "206D6F6E73696575 ( monsieu)",
            "count": 64,
            "length": 8
        },
        {
            "pattern": "0D0A0D (\r\n\r)",
            "count": 64,
            "length": 3
        },
        {
            "pattern": "C3A9746169656E74 (étaient)",
            "count": 64,
            "length": 8
        },
        {
            "pattern": "20656E636F726520 ( encore )",
            "count": 63,
            "length": 8
        },
        {
            "pattern": "697320 (is )",
            "count": 62,
            "length": 3
        },
        {
            "pattern": "6169742070617320 (ait pas )",
            "count": 62,
            "length": 8
        },
        {
            "pattern": "7468617420 (that )",
            "count": 62,
            "length": 5
        },
        {
            "pattern": "6F20746865 (o the)",
            "count": 62,
            "length": 5
        },
        {
            "pattern": "2E0D0A0D0A (.\r\n\r\n)",
            "count": 61,
            "length": 5
        },
        {
            "pattern": "732C20 (s, )",
            "count": 61,
            "length": 3
        },
        {
            "pattern": "6865207965 (he ye)",
            "count": 61,
            "length": 5
        },
        {
            "pattern": "6420746865 (d the)",
            "count": 60,
            "length": 5
        },
        {
            "pattern": "207769746820746865 ( with the)",
            "count": 59,
            "length": 9
        },
        {
            "pattern": "650D0A (e\r\n)",
            "count": 58,
            "length": 3
        },
        {
            "pattern": "6573206F66 (es of)",
            "count": 57,
            "length": 5
        },
        {
            "pattern": "2068697320 ( his )",
            "count": 57,
            "length": 5
        },
        {
            "pattern": "726573 (res)",
            "count": 57,
            "length": 3
        },
        {
            "pattern": "7C20202020202020 (|       )",
            "count": 56,
            "length": 8
        },
        {
            "pattern": "0D0A746865 (\r\nthe)",
            "count": 56,
            "length": 5
        },
        {
            "pattern": "2D2D2D2D2D2D2D2B2D (-------+-)",
            "count": 56,
            "length": 9
        },
        {
            "pattern": "626520 (be )",
            "count": 55,
            "length": 3
        },
        {
            "pattern": "207468652065617274 ( the eart)",
            "count": 55,
            "length": 9
        },
        {
            "pattern": "74206F6620 (t of )",
            "count": 55,
            "length": 5
        },
        {
            "pattern": "696F6E206F (ion o)",
            "count": 54,
            "length": 5
        },
        {
            "pattern": "617320 (as )",
            "count": 54,
            "length": 3
        },
        {
            "pattern": "0D0A0D0A2020546865 (\r\n\r\n  The)",
            "count": 53,
            "length": 9
        },
        {
            "pattern": "0A74686520 (\nthe )",
            "count": 53,
            "length": 5
        },
        {
            "pattern": "20746F2074 ( to t)",
            "count": 53,
            "length": 5
        },
        {
            "pattern": "696E672074 (ing t)",
            "count": 52,
            "length": 5
        },
        {
            "pattern": "6E206F6620 (n of )",
            "count": 52,
            "length": 5
        },
        {
            "pattern": "657265 (ere)",
            "count": 52,
            "length": 3
        },
        {
            "pattern": "686174 (hat)",
            "count": 52,
            "length": 3
        },
        {
            "pattern": "207265 ( re)",
            "count": 51,
            "length": 3
        },
        {
            "pattern": "3A204649472E2031 (: FIG. 1)",
            "count": 51,
            "length": 8
        },
        {
            "pattern": "696F6E206F66207468 (ion of th)",
            "count": 51,
            "length": 9
        },
        {
            "pattern": "2054686520 ( The )",
            "count": 50,
            "length": 5
        },
        {
            "pattern": "0D0A0D0A5B (\r\n\r\n[)",
            "count": 49,
            "length": 5
        },
        {
            "pattern": "6F6E206F6620746865 (on of the)",
            "count": 49,
            "length": 9
        },
        {
            "pattern": "6420696E20 (d in )",
            "count": 48,
            "length": 5
        },
        {
            "pattern": "6F6E206F66 (on of)",
            "count": 48,
            "length": 5
        },
        {
            "pattern": "6272 (br)",
            "count": 47,
            "length": 2
        },
        {
            "pattern": "0D0A0D0A2020202020 (\r\n\r\n     )",
            "count": 47,
            "length": 9
        },
        {
            "pattern": "206F660D0A ( of\r\n)",
            "count": 47,
            "length": 5
        },
        {
            "pattern": "7468652070 (the p)",
            "count": 46,
            "length": 5
        },
        {
            "pattern": "2EE2809D20 (.” )",
            "count": 45,
            "length": 5
        },
        {
            "pattern": "0A0D0A202054686520 (\n\r\n  The )",
            "count": 44,
            "length": 9
        },
        {
            "pattern": "7468652079 (the y)",
            "count": 44,
            "length": 5
        },
        {
            "pattern": "0A0D0A0D0A0D0A0D0A (\n\r\n\r\n\r\n\r\n)",
            "count": 44,
            "length": 9
        },
        {
            "pattern": "0A20 (\n )",
            "count": 44,
            "length": 2
        },
        {
            "pattern": "20202020207C207878 (     | xx)",
            "count": 44,
            "length": 9
        },
        {
            "pattern": "206265 ( be)",
            "count": 44,
            "length": 3
        },
        {
            "pattern": "7C20202020207C20 (|     | )",
            "count": 44,
            "length": 8
        },
        {
            "pattern": "746572 (ter)",
            "count": 43,
            "length": 3
        },
        {
            "pattern": "686572 (her)",
            "count": 43,
            "length": 3
        },
        {
            "pattern": "7374656C6C6174696F (stellatio)",
            "count": 43,
            "length": 9
        },
        {
            "pattern": "636F6E7374656C6C61 (constella)",
            "count": 42,
            "length": 9
        },
        {
            "pattern": "6C6C75737472617469 (llustrati)",
            "count": 42,
            "length": 9
        },
        {
            "pattern": "5F2C20322E0D0A2020 (_, 2.\r\n  )",
            "count": 41,
            "length": 9
        },
        {
            "pattern": "726520 (re )",
            "count": 39,
            "length": 3
        },
        {
            "pattern": "65206F (e o)",
            "count": 39,
            "length": 3
        },
        {
            "pattern": "2E0D0A2020616E6420 (.\r\n  and )",
            "count": 38,
            "length": 9
        },
        {
            "pattern": "6E7374656C6C617469 (nstellati)",
            "count": 38,
            "length": 9
        },
        {
            "pattern": "6C757374726174696F (lustratio)",
            "count": 37,
            "length": 9
        },
        {
            "pattern": "5B496C6C7573747261 ([Illustra)",
            "count": 37,
            "length": 9
        },
        {
            "pattern": "5F46 (_F)",
            "count": 37,
            "length": 2
        },
        {
            "pattern": "497361696168202020 (Isaiah   )",
            "count": 37,
            "length": 9
        },
        {
            "pattern": "6F66 (of)",
            "count": 36,
            "length": 2
        },
        {
            "pattern": "496C6C757374726174 (Illustrat)",
            "count": 36,
            "length": 9
        },
        {
            "pattern": "2E0D0A0D0A20205468 (.\r\n\r\n  Th)",
            "count": 36,
            "length": 9
        },
        {
            "pattern": "5F2C20362E0D0A2020 (_, 6.\r\n  )",
            "count": 35,
            "length": 9
        },
        {
            "pattern": "20696E2074686520 ( in the )",
            "count": 35,
            "length": 8
        },
        {
            "pattern": "20207C2020202020 (  |     )",
            "count": 34,
            "length": 8
        },
        {
            "pattern": "0D0A0D0A0D0A0D0A0D (\r\n\r\n\r\n\r\n\r)",
            "count": 34,
            "length": 9
        },
        {
            "pattern": "206F662074686520 ( of the )",
            "count": 33,
            "length": 8
        },
        {
            "pattern": "0D0A20 (\r\n )",
            "count": 27,
            "length": 3
        },
        {
            "pattern": "4672 (Fr)",
            "count": 26,
            "length": 2
        },
        {
            "pattern": "4C69 (Li)",
            "count": 26,
            "length": 2
        },
        {
            "pattern": "0A2020 (\n  )",
            "count": 25,
            "length": 3
        },
        {
            "pattern": "2020207C20202020 (   |    )",
            "count": 24,
            "length": 8
        },
        {
            "pattern": "616C20 (al )",
            "count": 23,
            "length": 3
        },
        {
            "pattern": "476C (Gl)",
            "count": 22,
            "length": 2
        },
        {
            "pattern": "0A0D (\n\r)",
            "count": 21,
            "length": 2
        },
        {
            "pattern": "6174656C6C697465 (atellite)",
            "count": 18,
            "length": 8
        },
        {
            "pattern": "207C202020202020 ( |      )",
            "count": 18,
            "length": 8
        },
        {
            "pattern": "20746F2074686520 ( to the )",
            "count": 17,
            "length": 8
        },
        {
            "pattern": "207C202020302E31 ( |   0.1)",
            "count": 17,
            "length": 8
        },
        {
            "pattern": "6F6265 (obe)",
            "count": 17,
            "length": 3
        },
        {
            "pattern": "61676E6974756465 (agnitude)",
            "count": 16,
            "length": 8
        },
        {
            "pattern": "0A0D0A (\n\r\n)",
            "count": 16,
            "length": 3
        },
        {
            "pattern": "64697374616E6365 (distance)",
            "count": 16,
            "length": 8
        },
        {
            "pattern": "74696F6E206F6620 (tion of )",
            "count": 15,
            "length": 8
        },
        {
            "pattern": "EFBFBDEFBFBD (��)",
            "count": 15,
            "length": 6
        },
        {
            "pattern": "6D61676E69747564 (magnitud)",
            "count": 15,
            "length": 8
        },
        {
            "pattern": "7468652073746172 (the star)",
            "count": 15,
            "length": 8
        },
        {
            "pattern": "6C6F62 (lob)",
            "count": 15,
            "length": 3
        },
        {
            "pattern": "2D2D2D2D2D2D2D2B (-------+)",
            "count": 15,
            "length": 8
        },
        {
            "pattern": "6572207468616E20 (er than )",
            "count": 14,
            "length": 8
        },
        {
            "pattern": "68652073616D6520 (he same )",
            "count": 14,
            "length": 8
        },
        {
            "pattern": "6D6574656F727320 (meteors )",
            "count": 14,
            "length": 8
        },
        {
            "pattern": "5F2020 (_  )",
            "count": 14,
            "length": 3
        },
        {
            "pattern": "736174656C6C6974 (satellit)",
            "count": 14,
            "length": 8
        },
        {
            "pattern": "2773206F72626974 ('s orbit)",
            "count": 13,
            "length": 8
        },
        {
            "pattern": "686520706C616E65 (he plane)",
            "count": 13,
            "length": 8
        },
        {
            "pattern": "74656C6573636F70 (telescop)",
            "count": 13,
            "length": 8
        },
        {
            "pattern": "46726F (Fro)",
            "count": 13,
            "length": 3
        },
        {
            "pattern": "5F4672 (_Fr)",
            "count": 13,
            "length": 3
        },
        {
            "pattern": "2077686963682061 ( which a)",
            "count": 12,
            "length": 8
        },
        {
            "pattern": "207468652073756E ( the sun)",
            "count": 12,
            "length": 8
        },
        {
            "pattern": "2074686520706C61 ( the pla)",
            "count": 12,
            "length": 8
        },
        {
            "pattern": "206D6F76696E6720 ( moving )",
            "count": 12,
            "length": 8
        },
        {
            "pattern": "6F6D20 (om )",
            "count": 12,
            "length": 3
        },
        {
            "pattern": "726F6D (rom)",
            "count": 12,
            "length": 3
        },
        {
            "pattern": "7065637472756D20 (pectrum )",
            "count": 12,
            "length": 8
        },
        {
            "pattern": "2073746172732C20 ( stars, )",
            "count": 12,
            "length": 8
        },
        {
            "pattern": "616C74686F756768 (although)",
            "count": 12,
            "length": 8
        },
        {
            "pattern": "706C616E65742773 (planet's)",
            "count": 12,
            "length": 8
        },
        {
            "pattern": "20476C ( Gl)",
            "count": 12,
            "length": 3
        },
        {
            "pattern": "6520706C616E6574 (e planet)",
            "count": 12,
            "length": 8
        },
        {
            "pattern": "6520737461727320 (e stars )",
            "count": 12,
            "length": 8
        },
        {
            "pattern": "207468652073616D ( the sam)",
            "count": 12,
            "length": 8
        },
        {
            "pattern": "206D617920626520 ( may be )",
            "count": 11,
            "length": 8
        },
        {
            "pattern": "697374616E636520 (istance )",
            "count": 11,
            "length": 8
        },
        {
            "pattern": "2E205F (. _)",
            "count": 11,
            "length": 3
        },
        {
            "pattern": "205F46 ( _F)",
            "count": 11,
            "length": 3
        },
        {
            "pattern": "747765656E207468 (tween th)",
            "count": 11,
            "length": 8
        }
    ]
}

Sunday, January 12, 2014

A leap in artificial intelligence ? Continued 3

In this saga about our (yours and mine)  

Artificial intelligence that learns from associated parallel streams of information

today's reflexions are about the order in which we would like this AI to learn things.

Despite that we still don't know much about what this new AI should do (see previous post) the show must go on. (But feel free to still add your contributions to my previous post).

Spending a little bit of grey energy on what this AI should learn first might by handy.




Ever thought about why we use smiley's?

Have a look at this short video showing twice the Dutch phrase "Ja, ja" (in English: "Yes, yes" as you might have guessed):


The first case is a double affirmation / agreement, the second on the other hand means something like: "talk as much as you want, but I don't believe you".

Another striking example has been posted by +Maya Davis in mai 2013, about the pronunciation of Shakespeare's work.

In the field of computational linguistics we know very well that interpreting written text is confronted with the issue of highly complex structural relationships (as opposed to linear order). Making unambigous interpretation extremely difficult.

Obviously something important is lacking in written text:

The extra things we have in spoken text: pitch, stress, pauses, etc.
And body language if we can see the person.


But we do understand written text, don't we? So what's up?

My interpretation is that when we read we imagine what the melody, intonations etc are (or are supposed to be). Sometimes we imagine the corresponding body language also.
Smiley's are one way to fill in the gaps in expressiveness of text compared to the spoken word.

In evolution?
As a baby, did you start writing or seeing and listening?
Language could be as old as 100,000 years, where as writting appeared around 3200 BCE.
Whow, that were two easy ones!


What do you think our AI should start with:

the texts available everywhere
or
rich speech recognition?



What do you think to see when looking at a picture?
Something in two dimensions or do you make the shift to 3D?
If so, why can you do it?

Does M.C. Esher ring a bell?

Drawing by M.C. Esher

In images like the one above, personally I shift to a series of 3D images. Changing the image with every part of the drawing I'm focusing on. No way for me to make it into a single coherent picture.

And classical video (not 3D)? Do you "see" depth? What do you use to interpret a 2D video into a 3D live like experience?

Like for the language: What do you think the AI should start with:

2D
or
3D vision (binocular or perhaps even N-ocular)?


Finally for the vision component: what would you like the AI to see here:

post from +Donavon Urfalian 

An owl
or
a bunch of fruit and vegetables?
or
Both? In which order?



The last thing for today is a little clarification.

Although I frequently try to make you think about how we, the human beings, do things, it doesn't necessarily imply that our future AI has to be biologically inspired.

The purpose is to dress a holistic view on the richness of our perception, the implication of combined sensory input and the need to rely on previously learned things for current interpretations (and actions).



Don't forget to answer the three questions above. At least for yourself. And if you feel the need to express it in a comment, you're welcome.

First chapter Previous chapter Next chapter

Wednesday, January 8, 2014

Personal Archive DIgitization Project

When you're reading this, the final #PADIP count-down has reached the 0.





When this count-down started at 25



there was a pile left of 25 cm of documents to be scanned.

The last 25 cm pile of documents to be scanned.


So what is this Personal Archive DIgitization Project  ?

Several years ago I decided that it was time to get rid of the numerous piles of papers (and boxes) I conserved since the age of 14. Archives of 43 years.

Before that time there is not so much, just a few things from the archives of my parents that made it until now. Some of which go back to my birth year. Like this one:

The card my mother made with the evolution of my weight since my birth.


I'm talking about the scanning of 14000+ documents (estimated at 23000 pages) and some 6000 photo's.
My first photo's are from March 10, 1966: several shots from the live television transmission of the wedding of Princess Beatrix (our former queen).

Most of the photo's were scanned from the negatives by an external company though (and there are still some 400 photo's and 300 diapositives left to do).

These figures are very low compared to Google's millions of book scans, but as a personal project they are considerable. It least it took me several years working on it in my already very busy spare time.


And nostalgic, I conserved two folders of paper documents of a particular emotional value.


Phase 1 of PADIP : get rid of (most of) the paper (and make it available for recycling). => Done


OK, now I have a hard disk with these files (and backups of course), and an operating system that can easily find what I'm looking for in the documents from the typewriter and printer ages. Thanks to a relatively performant OCR.
For the handwritten material the scans have to rerun through the next generation of OCR software. Actually the only clues are in the meaningful file names.

Parallel to these scanned documents there are of course the digital age sources, and some digital record management headaches ahead. These digital archives start in 1986 and are continuously growing. Numbers? No precise idea yet but a rough estimation brings me 20000 emails, 10000 documents and 16000 digital photo's.

There is also some structured data available. Just two small examples: since 2008 (my first iPhone) every 5 seconds my position is recorded when I'm on the road. That are 1800+ trails. And a database of my 1500 books with the date I bought them, price, pages, dimensions etc. (Yet another scan project ?)

Perhaps I should ask Google to be able to download my browser history, and the telephone companies to download my cellular positions (from before 2008). It would be nice to have them also. NSA has them, so why not me?


So now I have my own personal Big Data. But most unstructured so not very useful, yet. But the next phase in this project will solve that.


In this post I was talking about information extraction, and in another series of posts about artificial intelligence. And for the past 20 years I've been working on semantic network technologies (comparable to Google's knowledge graph).

Got the global picture?

Fine.

The trick will be to build a coherent picture of all the information available, correcting OCR errors, defining location and time of photo's, distinguishing meetings I've been to from the ones I've been invited for but didn't attend, etc.

The challenge is not in getting the data, but in creating to software to get the right data, or better, getting the data right. Doing this manually is not an option.

This personal Big Data set will provide "food for thought" for the new AI (with a lot of other stuff of course).
And as three languages (Dutch, French and English) (and snippets of Spanish, Italian and Polish) are scattered around in this set, it will also start its learning process to be able to provide in return a language teaching capability for humans as illustrated in the short (sort of prototype app) video.



Challenging isn't it?



#PADIP #ArtificialIntelligence #InformationExtraction #PersonalData

Edit 2014-01-13 Changed digitalization to digitization

Thursday, December 12, 2013

A leap in artificial intelligence ? Continued 2.


Picking up the thread of previous post I was asking myself the question:

Does our future artificial intelligence that learns from associated parallel streams of information can learn everything needed to follow our trail?

In other words is there any input (and output) that we have which is not easy to capture by our AI or cannot easily be associated with other kinds of input.

Scratch, scratch.  (OK AI, FYI: these are the words I use to describe the sound when I move my fingertips in a back and forth movement on the top of my head). (For future use.)

I can imagine 2 of them. You?

The first one that pops up is smell and taste. For me those two are so closely related that I put them at the same level. When preparing my food I never taste it. Smelling it gives me the exact taste it will have and allows me to adjust the seasoning. Feel free to consider them as separated.
We already have a lot of gas sensor networks (and could easily extend them). But they cover only the detection of small part of the molecules in the air.
And even if they would, I'm far from sure that there is enough data available to learn the step from a set of detected molecules to our 3 way classification of odors (i.e. smells good, stinks or smells like a hospital).
We have of course our fruit esters which are single molecules that define what we smell and which are well described, but are there gas detectors for them?

update: thanks +snakeappletree white-lightning-gate : Smell and taste from organic biological sensors connected to a symbiotic computer chip and synapse brain are already in development.



I hope you're all aware that we are one big walking set of touch sensors. Because that's the second one.
Now think of all the words you currently use to describe how something feels (physically).
Next associate the kind of sensors our AI must have to detect what might correspond to these words.
Sure you came up with the thermometer, the high resolution closeup camera (microscope) and several others.
But what about movement of the air from the wing beat of the butterfly the small hairs on my bare arm are detecting (the one that causes a tornado on the other side of the world)? Our Anemometers? Nope. Indirectly a camera perhaps.
The itch of a mosquito sting?

Of course there are other things that might be difficult to learn for our AI (like our emotions - although or body language does express a lot about our emotional state) but feel free to comment on important input I didn't mention.

I can hear you ask (OK AI: Try to make sense of of that): Why is he talking about these subjects?

I virtually like to dive into the deep blue from a helicopter view. Trying to see where creativeness is needed to fill the gaps. Having a reasonable feeling where more head scratches are needed. Having a holistic view.
Trying to avoid the "Oh S.." reaction at an advanced stage.

In case you noticed that terms like "problem" or "cannot" do not occur very often in my writings: the reason for it is that it's not in the beast to think in these terms. Especially when innovating.



Let's move on to the second part of this post. The part where your active participation will be appreciated.


Before thinking about how we could make our


artificial intelligence that learns from associated parallel streams of information


it is useful to think a bit about what you want this AI to do.

You know how to use the comments, so go ahead. But read the rest of this post first.

Just as +carey g. butler asked in a comment on my previous post "What is learning?", you might ask yourselves "What is artificial intelligence?"

I cannot give you an answer. Can you?
+Charles Isbell gives a few examples he likes in his talk 2 years ago. And +Luciano Floridi has his view also. And of course there are the wikipedia pages for AI and strong AI.
Using your favorite search engine, you will find other "definitions" you might like (or not).

Personally I tend to consider this domain to be a continuum ranging from the Luciano's not so smart dishwasher robot, through our spelling checkers, the smart camera's for collision detection, voice search, Google's self driving car up to the yet to come cognitive general AI.
The thing they have in common is that there is a, direct or not, interaction / relation with human beings.


Almost independent of what you might consider being AI, I like to separate the "do's" of such an AI into at least 4 groups of targets.

- for individuals
- for small groups of people that know each other
- for large groups of people that don't know each other
- for itself

The reason for this distinction is that the "how to's" might be different. But it is by no means a very strict separation. We have always lived in some sort of society which implicates interactions of an individual with other entities (other individuals or organizations and objects). It is only a handy classification.


You and me of course.
But think also about older and younger generations. Your parents, grand-parents, your kids and the yet to be born. They probably have a view quite different from yours.
Your personal butler answering your questions, doing things for you, informing you. Teaching you.
How much of your personal information are you willing to provide to let the AI help you with your daily things? How much do you give already to services you're using every day (with not much in return)? And what would you be willing to give if there is a guarantee that nobody else will get that information (this is possible see e.g. Wouter Teepe: Reconciling Information Exchange and Confidentiality. A Formal Approach)?


Your family, friends and professional contacts.
Negotiating best moments for non disruptive phone calls / hangouts, place and time for business meetings. Interesting movie to watch together tomorrow evening.
Your home negotiating with the electricity company the price of electricity (during a particular period of the day) if the dishwasher can be convinced to do its job at non-peak hours.


Town, country and world (universe ?)
The same electricity company as above negotiating with hundreds of thousands of houses. The chatting traffic lights (with other ones similar to themselves and with strange objects passing by on their two or four  wheels). Sorry, my mistake, there will be no traffic lights anymore in the (near) future.
Ding, dong. Garbage bin 2, get yourself out, I'll be there in 7.46783 minutes.
Dear B.O., ten years from now the flooding risk in this area will have increased with 13%. The associated loss of human lives and economic damage will have increased with 27%. Starting the following actions within a year will reduce the damage to an increase of only 9% and will cost ...


The AI itself and its cousins
Without going through an exhaustive list your can find everywhere in the literature, I feel comfortable with two main categories of things the AI should do for itself:
- staying alive up and running
- improve performance

Now go wild and think about the wide variety of implications these two simple drives might induce.
And of course can they be learned?



Here is the game:

If you want to play, write a short comment on what you want such an AI to do.
Try to keep it short and synthetic, enough to let me and the other readers get the idea.

I'll do my best to keep this post updated with your participation below this line.


First chapter Previous chapter Next chapter


Update 2013-12-14: Typo.

=============================================

You and me of course.
- Re affirm my own relevance in life and socialise with me in ways that suit my mood, varying from parent to disciple, teacher to student, friend to lover; companionship custom designed in ways that humans of our social indoctrination culture are incapable.  +snakeappletree white-lightning-gate


Your family, friends and professional contacts.



Town, country and world (universe ?)



The AI itself and its cousins
- I would like AI to grow naturally and have a 'good childhood'. When we move too fast, we make mistakes. Our growth should be deliberate and guided by our moral and ethical senses (compasses). Only in an environment that contains them will that which we create avoid being a danger to ourselves. +carey g. butler 
- I would like AI to understand well and be wise. AI should also be aware of the natural tendencies of any particular distribution involving life or choice to cleave itself into an elite of hegemonious control to feed itself upon the rest of the population (even Cancer shares these attributes). +carey g. butler 

Tuesday, December 10, 2013

A leap in artificial intelligence ? Continued.

About a month ago I wrote the first part of my thoughts about how to make the so much wanted leap in artificial intelligence.
Today I feel the need to release the next part.


Ever read a book? You probably did.
Ever speaking the text out loud? Perhaps when you were a kind.
Ever silently spoke it in your head? I did and do (even when writing this). And you?

Why?


Do you remember the time when your mother or father told you a story before going to sleep?
You probably don't remember what happened in your mind at those moments (the recall of our memories is not supposed to go that far back in time).  But imagine what might have happened.
I think I made it into a film. Loosing part of what was told as my internal story follows its own scenario, and merging both stories again a few moments later.
Ever lost attention during a presentation? What happened?

Why?


The "man in the box" jumping out or a suddenly moving living statue induces strong reactions.

Why?


Look at this question: r thr n tp's lft n th txt y wrt?

Why?


Before you will be attempting to answer these four why questions, a quick tour on our actual AI techniques.
Most (if not all) machine learning techniques are based on a simple idea:
- take a set of similar things (text, images, ...), labelled or not
- do some calculations with them (statistics, neural network training, ...)
- apply the results to another set of similar things


My previous post was centered around the idea of parallel treatment of different kind of things and associating these:
- written and spoken word
- image of an object and the word for it
etc.

But do we learn from static things?

Our environment is all but static. Although there are static objects in it, we aren't. Which makes the whole dynamic.

When we started learning as a baby, quietly observing our world from our cradle, what do we see? Things that don't move and things that do. Some things move when you touch them. Everything moves when you're lifted out.

I think our capability of very quickly identifying static things comes from our experience in identifying things in a dynamic environment. Not the other way around.
Our mind is trained to recognize things in a stream of information. And when provided with static things (images in our childhood book e.g.) we make them dynamic.
When looking at a painting of a still live do you see it in 2D or do you make it a 3D and sort of feeling the objects?

Now return to the why's. Apply the need for a stream of information to them. Expecting things to move or not.
Does it seem to make sense?

The typo's example might be the most difficult. But if you look at a piece of text as a flow of words (we don't read the individual characters). Anticipating the next words. Filling in the lacking words, replacing the wrong ones, not seeing the misspellings. (btw: did you pronounce the question?)
A real life experience I had several years ago is worth mentioning. For a knowledge management project we did several interviews. They were all recorded. During the transcription we discovered that one person, during the 90 minutes interview, never had finished a sentence. We didn't noticed that until the transcription. We must have made up the end of the sentences while he was talking.

Back to our artificial intelligence.

Imagine another kind of machine learning. One that learns from a flow of information. Looking at the delta's between two moments in time (video, sound but also text).
Instead of learning cat (and human) faces from independent still images it will learn what cats are from sequences of images. Their 3D appearance, their degrees of liberty, how they move etc. Probably even reaction patterns. Once this is acquired it is easy to "create" a front view: the face of the cat.
Objects that do not change in position can be viewed with a moving camera (or more then one camera).
For images the base techniques exist, nothing new. We only have to ...

use it in our actual artificial intelligence toolbox. This toolbox might be (almost) good enough but we should apply it for learning different things (and tune a few things).
Summarized this might give


Artificial intelligence that learns from associated parallel streams of information.


Does your mind starts wandering, thinking perhaps about the "how to"?

Fine.

Sit back and relax.


First chapter Next chapter


2013-12-11 Edit: typos