cjvt-valency/dip_src

Izdelava vezljivostnih vzorcev za slovenske glagole

Študent: Kristjan Voje

Mentor: prof. Marko Robnik Šikonja Somentorica: dr. Apolonija Gantar

Requirements:

• Linux (built and tested on Ubuntu 16.04),
• python3

Quick workspace preparation

# Clone the repo.
$ git clone https://voje@bitbucket.org/voje/diploma.git

# Prepare the data.
$ cd ./data
$ unzip data.zip

Virtualenv recommended. $ sudo pip3 install virtualenv.

$ cd ./script
$ virtualenv -p /usr/bin/python3 venv

# A folder venv with python libraries will appear.  
# To activate the virtual environment:  
$ source ./venv/bin/activate

# To deactivate: 
$ deactivate

Now, to build the python packages:

# Polyglot dependencies
$ sudo apt-get install libicu-dev
# NOTE: installing this broke my Arch system.
# If on Arch, use the AUR version (it's supposed to be safe).

$ cd ./script
$ pip3 install -e .

# Polyglot downloads
$ polyglot download morph2.sl

To test the installation, fire up a python3 shell (while in virtualenv) and:

import valency

Nodejs environment

There's a compiled front end client in the git repo. In case you want to change anything, you'll need to set up the development environment:

• install nodejs (default Ubuntu versions are usually behind),
• check $ npm -v, if you don't have it, install npm,

$ npm install vue-cli
$ cd ./vue_frontend
$ npm install
$ npm run dev  # for development
$ npm run build  # for production

You will also need to change some path variables if you'll want to access the backend api from vue development server. See chapter "Web app deployment".

MongoDB

Set up the database on the system

install MongoDB on Linux

The MongoDB instance stores its data files in /var/lib/mongodb and its log files in /var/log/mongodb by default, and runs using the mongodb user account. You can specify alternate log and data file directories in /etc/mongod.conf. See systemLog.path and storage.dbPath for additional information.

If you change the user that runs the MongoDB process, you must modify the access control rights to the /var/lib/mongodb and /var/log/mongodb directories to give this user access to these directories.

Check if it's working.

$ sudo service mongod start / stop / restart
$ tail -n 30 /var/log/mongodb/mongod.log

# If you want the mongo shell on local machine:
$ mongo --host 127.0.0.1:<27017> # Check net.port in /etc/mongod.conf

Security

Create admin and user. Localhost only. sedurity man Basically, create an admin user then user that admin to create a normal user. Something like this:

use admin

db.createUser(
  {
    user: "admin_name",
    pwd: "admin_pass",
    roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
  }
)

# Restart mongod.

use texts

db.createUser(
  {
    user: "user_name",
    pwd: "user_pass",
    roles: [ { role: "readWrite", db: "texts" } ]
  }
)

# Restart mongod.

# Also useful
db.dropUser("username")

Then go to /etc/mongod.conf and add this: Do not use tabs! mongod won't start.

security:
 authorization: enabled

You need to use <db>, db.auth("username", "pass") to have access.

DB migration

I installed mongo on remote with same user accounts as on local.

# On local.
$ mongodump --db texts --host mongodb1.example.net --port 3017 --username user --password "pass" --out ./file.db

# Rsync it over to remote.
# On remote.
# Need to turn off authorization for the drop part.

$ mongorestore --db texts --drop --port <port> ./<dbfolder>

DB collections, needed for web app

• v2_senses,
• v2_senses_map,
• v2_users,
• v2_user_tokens
• sskj

Web app deployment

We need to set flask to serve the vuejs frontend:

# in ./script/flask_app/app.py
app = Flask(
    __name__,
    static_folder="./vue/dist/static",
    template_folder="./vue/dist"
)

Set the correct CORS options.

Might need to reinstall the package. Make sure you're in python virtualenv. $ pip3 install -e .

We also need to tell vuejs client to look for the api on server's address.

// in script/vue_frontend/src/main.js
api_addr: "http://<my_server_addr>:5004",

Compile frontend client:

$ cd ./script/vue_frontend
$ npm run build
$ rm ../flask_app/vue -rf
$ cp ./dist ../flask_app/vue -r

After preparing frontend, backend and database, run the command:

$ cd ./script
$ ./autostart.sh

When fixing things in dev environment:

## Backend
# git clone everything
$ cd script

# unzip some zipped files
$ ./sherpa --unpack

# virtualenv
$ source venv/bin/activate
$ pip3 install -e .

# start a mongod instance
$ sudo systemctl start mongod

# start the backend server
$ ./autostart.sh --debug

# you can watch the logs with
$ tail -f log/main.log


## Frontend
# set the right config file (path to api)
$ ./sherpa --frontend_config dev
$ cd vue_frontend
$ npm run dev


## Done making changes
# build frontend for production with the right config files
$ ./sherpa --build_vue

# if you've made changes to static .pickle files
$ ./sherpa --pack