From What to How: what is seen from here is not seen from there.

It is well know citate of Ariel Sharon about Israel leaving Gaza. This move looks controversial taking into consideration that he was the head of troops conquested it and Sinai.

I'm not in a position to judge how true is this in politics. In programming it is 200% true.

No matter how good is your plan, it will dramatically change when you'll start to implement it. And the cause is exactly this: you see the thing from a different point.

Amendments to the previous plan:

Functionality to implement

• Abilities to train: learning to read and listen will not be successful without learning to write and speak. Need to add 2 more flows:
  • Listen & Repeat - sentence-by-sentence or even word-by-word slow and then faster and faster
  • Dictation

Data

• User profile data:
  • UI language
  • Subjects - in learning and available
  • Current subject and lessons available for the next learning session
• Very useful flow - user adds a text into the First Read and system builds a new lesson out of it on the fly.

Technology to use

• It looks like Firebase is a good starting point for PWA hosting
• Store various relations of the data in Neo4j graph database
  • Graph database is designed specifically for storing relations
  • Query language looks very similar to one of MongoDB
  • What will happen to query performance when we'll need to query 2 different servers while collecting data for a lesson?

To be continued ...

From What to How: language learning app - initial analysis

This project is really huge.
Now I'm going to analyze it's first release with very limited functionality.

Functionality to implement

• Abilities to train:
  • Read
  • Listen
• Flow:
  • Show a text and read it to the user highlighting read portion with one color and a word to read next with another color
  • Prompt user to select unknown words
  • Show explanations for each word from publicly available dictionaries one at a time
  • Select the next texts as another uses of unknown words and add them to the reading queue
  • For the new user there are a number of simple texts in the queue.
  • If queue becomes empty - user knows each word in the text - present a list of topics to select from.
  • Ideally new text should not contain more than 10% of new words.

Data

• Corpus of short texts indexed by:
  • Words
  • Morphems (when possible)
  • Idioms (when possible)
• Word explanations collected on demand
• Data about user's performance:
  • Frequency, time spent and payload of lessons
  • User's dictionary: words what never were marked as unknown or stopped to be marked as such (with count of both states or list of docs)
  • Attention: how many words are marked as unknown for the first time after a number of appearences
  • Learning effectiveness: how many times is a word marked as unknown?

Technology to use

• A progressive web app (PWA) storing user's data on user's machine.
  • same code for desktop/tablet/mobile
  • no installation and upgrades
  • 100% private for the user
  • user == user agent(browser or mobile device)
• Store corpus and word explanations in hosted MongoDB
  • MongoDB supports regular expression as data type.
  • I know how to do it in SQL. Now it's time to learn NoSQL 😉.
  • Hosted (not self-hosted) is a necessity. I'm not ready to run my own web server yet.
• Use clustering for morphem and idiom extraction
  • It should be able to accept hits from users (with grain of salt of course)
• Use GraphQL for client-server interaction
  • Easier to grow with the progect than REST
  • Steeper learning curve
• Use Text-to-Speech (TTS) functionality available in browser
  • No need to store audio
  • Easy synchronization with text highlight
• For the beginning implement backend in Python
  • Python is the most equiped language for ML
  • It supports all I need in backend now

Need to learn

• Asynchrony implementation in Python
• GraphQL
• PyMongo
• Motor?
• Graphene or Ariadne?
• Tornado?

Ready snippets

• First Read

To be continued ...

git - building from source on CentOS 6

I wanted to revise my knowledge and skills in git.

Taking a GitHub course sounded like a good idea, but git is invasive into user/credentials/config matters so I didn't dare to do it from my main user.

Old CentOS 6 machine collects dust under my desk and no one cares about it. Why cannot I make my git lessons on it?

$ git --version
$ git version 1.7. ..

(cannot recall the last digit)

Upgrade does not help - the official RHEL repository only has outdated version.
Should I add another repository and try upgrading from there?.. Or perhaps just try building from source?

Let's get source:

$ mkdir gitsource
$ cd gitsource
$ git clone https://github.com/git/git

Now I have the source in gitsource/git. Googling for build instructions leads me here and I do exactly as it reads:

$ yum install curl-devel expat-devel gettext-devel openssl-devel zlib-devel
$ yum install gcc perl-ExtUtils-MakeMaker

This provides me with old as my life gcc version 4.4.7 and all rusty rest.

Let's start building:

$ cd git $ make prefix=/usr/local/git all

It runs... Uuups! git-http-fetch error 1

Googling the error. Answers look unrelated.

Looking up the terminal. Compiler goes to anaconda directory at different user. What is in the PATH? Exactly! The path to anaconda goes first. Remove it.

No more errors.

$ make prefix=/usr/local/git install
$ echo "export PATH=$PATH:/usr/local/git/bin" >> /etc/bashrc

It's not the best idea to change global PATH, just have stumbled at it, but at least adding it at the end.

Checking:

$ git --version
$ git version 2.24.GIT

Now I can start learning.

The very beginning

I decided to begin with static part of the site.

When one makes a business or thematic website he tries to choose a backend that can cover all necessary features and then builds everything in it.

I'm going to try and demonstrate many different things and would like to place each in one of its best environments. It means I need different backends.

So I'm beginning with simplest one - static.

There are a number of reasons to make part of the site static:

• I don't build websites on daily basis. Last 12 years I mostly code in C++ and need to refresh my knowledge and skils in web design and developing.
• My domain registrator allows me to place a static site on their server for free. I find it a good idea to begin with free option and add paid ones when I really need them and know exactly what I need.
• It's interesting to explore how much can one do in confines of a static site.

I started with refreshing my HTML, CSS and client-side JavaScript.

When you are entering a new or well-forgotten field, the best first step is to find a good overview.

I strongly believe that "Get Started" articles are not the best starting point. You need to see the whole thing or at least as much of it as possible form a "helicopter perspective" before diving into it head first.

I was lucky to find rather good HTML/CSS overview in this tutorial. I was so impressed that asked the author if I can translate it into Russian and Bulgarian. Unfortunately I've not received any answer yet.

In the same time I tried to find a designer for my site. Technical details are my cup of tea, but design... I'm a painter with the stress on pain :(

Unfortunately this position is still open. People prefer providing laymans with turn-key solutions. When I tell I need design without technical implementation their smiles fade...