cjvt-valency/src_diploma/README.md

231 lines
5.2 KiB
Markdown
Raw Normal View History

2019-03-07 08:00:01 +00:00
# Izdelava vezljivostnih vzorcev za slovenske glagole #
Študent: Kristjan Voje
Mentor: prof. Marko Robnik Šikonja
Somentorica: dr. Apolonija Gantar
### Requirements:
* Linux (built and tested on Ubuntu 16.04),
* python3
## Quick workspace preparation
```bash
# Clone the repo.
$ git clone https://voje@bitbucket.org/voje/diploma.git
# Prepare the data.
$ cd ./data
$ unzip data.zip
```
Virtualenv recommended. `$ sudo pip3 install virtualenv`.
```bash
$ cd ./script
$ virtualenv -p /usr/bin/python3 venv
# A folder venv with python libraries will appear.
# To activate the virtual environment:
$ source ./venv/bin/activate
# To deactivate:
$ deactivate
```
Now, to build the python packages:
```bash
# Polyglot dependencies
$ sudo apt-get install libicu-dev
# NOTE: installing this broke my Arch system.
# If on Arch, use the AUR version (it's supposed to be safe).
$ cd ./script
$ pip3 install -e .
# Polyglot downloads
$ polyglot download morph2.sl
```
To test the installation, fire up a python3 shell (while in virtualenv) and:
```python
import valency
```
## Nodejs environment
There's a compiled front end client in the git repo.
In case you want to change anything, you'll need to set up the development environment:
* install nodejs (default Ubuntu versions are usually behind),
* check `$ npm -v`, if you don't have it, install npm,
```
$ npm install vue-cli
$ cd ./vue_frontend
$ npm install
$ npm run dev # for development
$ npm run build # for production
```
You will also need to change some path variables if you'll want to access the backend api from vue development server.
See chapter "Web app deployment".
## MongoDB
### Set up the database on the system
[install MongoDB on Linux](https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/)
> The MongoDB instance stores its data files in /var/lib/mongodb and its log files in /var/log/mongodb by default, and runs using the mongodb user account. You can specify alternate log and data file directories in /etc/mongod.conf. See systemLog.path and storage.dbPath for additional information.
>
> If you change the user that runs the MongoDB process, you must modify the access control rights to the /var/lib/mongodb and /var/log/mongodb directories to give this user access to these directories.
#### Check if it's working.
```bash
$ sudo service mongod start / stop / restart
$ tail -n 30 /var/log/mongodb/mongod.log
# If you want the mongo shell on local machine:
$ mongo --host 127.0.0.1:<27017> # Check net.port in /etc/mongod.conf
```
#### Security
Create admin and user. Localhost only.
[sedurity man](https://docs.mongodb.com/v3.4/tutorial/enable-authentication/)
Basically, create an admin user then user that admin to create a normal user.
Something like this:
```
use admin
db.createUser(
{
user: "admin_name",
pwd: "admin_pass",
roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
}
)
# Restart mongod.
use texts
db.createUser(
{
user: "user_name",
pwd: "user_pass",
roles: [ { role: "readWrite", db: "texts" } ]
}
)
# Restart mongod.
# Also useful
db.dropUser("username")
```
Then go to `/etc/mongod.conf` and add this:
Do not use tabs! mongod won't start.
```bash
security:
authorization: enabled
```
You need to `use <db>`, `db.auth("username", "pass")` to have access.
#### DB migration
I installed mongo on remote with same user accounts as on local.
```bash
# On local.
$ mongodump --db texts --host mongodb1.example.net --port 3017 --username user --password "pass" --out ./file.db
# Rsync it over to remote.
# On remote.
# Need to turn off authorization for the drop part.
$ mongorestore --db texts --drop --port <port> ./<dbfolder>
```
#### DB collections, needed for web app
* v2_senses,
* v2_senses_map,
* v2_users,
* v2_user_tokens
* sskj
## Web app deployment
We need to set flask to serve the vuejs frontend:
```python
# in ./script/flask_app/app.py
app = Flask(
__name__,
static_folder="./vue/dist/static",
template_folder="./vue/dist"
)
```
Set the correct CORS options.
Might need to reinstall the package. Make sure you're in python virtualenv.
`$ pip3 install -e .`
We also need to tell vuejs client to look for the api on server's address.
```javascript
// in script/vue_frontend/src/main.js
api_addr: "http://<my_server_addr>:5004",
```
Compile frontend client:
```bash
$ cd ./script/vue_frontend
$ npm run build
$ rm ../flask_app/vue -rf
$ cp ./dist ../flask_app/vue -r
```
After preparing frontend, backend and database, run the command:
```bash
$ cd ./script
$ ./autostart.sh
```
## When fixing things in dev environment:
```bash
## Backend
# git clone everything
$ cd script
# unzip some zipped files
$ ./sherpa --unpack
# virtualenv
$ source venv/bin/activate
$ pip3 install -e .
# start a mongod instance
$ sudo systemctl start mongod
# start the backend server
$ ./autostart.sh --debug
# you can watch the logs with
$ tail -f log/main.log
## Frontend
# set the right config file (path to api)
$ ./sherpa --frontend_config dev
$ cd vue_frontend
$ npm run dev
## Done making changes
# build frontend for production with the right config files
$ ./sherpa --build_vue
# if you've made changes to static .pickle files
$ ./sherpa --pack
```