231 lines
5.2 KiB
Markdown
231 lines
5.2 KiB
Markdown
# Izdelava vezljivostnih vzorcev za slovenske glagole #
|
|
|
|
Študent: Kristjan Voje
|
|
|
|
Mentor: prof. Marko Robnik Šikonja
|
|
Somentorica: dr. Apolonija Gantar
|
|
|
|
### Requirements:
|
|
|
|
* Linux (built and tested on Ubuntu 16.04),
|
|
* python3
|
|
|
|
## Quick workspace preparation
|
|
```bash
|
|
# Clone the repo.
|
|
$ git clone https://voje@bitbucket.org/voje/diploma.git
|
|
|
|
# Prepare the data.
|
|
$ cd ./data
|
|
$ unzip data.zip
|
|
```
|
|
|
|
Virtualenv recommended. `$ sudo pip3 install virtualenv`.
|
|
```bash
|
|
$ cd ./script
|
|
$ virtualenv -p /usr/bin/python3 venv
|
|
|
|
# A folder venv with python libraries will appear.
|
|
# To activate the virtual environment:
|
|
$ source ./venv/bin/activate
|
|
|
|
# To deactivate:
|
|
$ deactivate
|
|
```
|
|
|
|
Now, to build the python packages:
|
|
```bash
|
|
# Polyglot dependencies
|
|
$ sudo apt-get install libicu-dev
|
|
# NOTE: installing this broke my Arch system.
|
|
# If on Arch, use the AUR version (it's supposed to be safe).
|
|
|
|
$ cd ./script
|
|
$ pip3 install -e .
|
|
|
|
# Polyglot downloads
|
|
$ polyglot download morph2.sl
|
|
```
|
|
|
|
To test the installation, fire up a python3 shell (while in virtualenv) and:
|
|
```python
|
|
import valency
|
|
```
|
|
|
|
## Nodejs environment
|
|
There's a compiled front end client in the git repo.
|
|
In case you want to change anything, you'll need to set up the development environment:
|
|
|
|
* install nodejs (default Ubuntu versions are usually behind),
|
|
* check `$ npm -v`, if you don't have it, install npm,
|
|
```
|
|
$ npm install vue-cli
|
|
$ cd ./vue_frontend
|
|
$ npm install
|
|
$ npm run dev # for development
|
|
$ npm run build # for production
|
|
```
|
|
You will also need to change some path variables if you'll want to access the backend api from vue development server.
|
|
See chapter "Web app deployment".
|
|
|
|
## MongoDB
|
|
### Set up the database on the system
|
|
[install MongoDB on Linux](https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/)
|
|
|
|
> The MongoDB instance stores its data files in /var/lib/mongodb and its log files in /var/log/mongodb by default, and runs using the mongodb user account. You can specify alternate log and data file directories in /etc/mongod.conf. See systemLog.path and storage.dbPath for additional information.
|
|
>
|
|
> If you change the user that runs the MongoDB process, you must modify the access control rights to the /var/lib/mongodb and /var/log/mongodb directories to give this user access to these directories.
|
|
|
|
#### Check if it's working.
|
|
```bash
|
|
$ sudo service mongod start / stop / restart
|
|
$ tail -n 30 /var/log/mongodb/mongod.log
|
|
|
|
# If you want the mongo shell on local machine:
|
|
$ mongo --host 127.0.0.1:<27017> # Check net.port in /etc/mongod.conf
|
|
```
|
|
|
|
#### Security
|
|
Create admin and user. Localhost only.
|
|
[sedurity man](https://docs.mongodb.com/v3.4/tutorial/enable-authentication/)
|
|
Basically, create an admin user then user that admin to create a normal user.
|
|
Something like this:
|
|
```
|
|
use admin
|
|
|
|
db.createUser(
|
|
{
|
|
user: "admin_name",
|
|
pwd: "admin_pass",
|
|
roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
|
|
}
|
|
)
|
|
|
|
# Restart mongod.
|
|
|
|
use texts
|
|
|
|
db.createUser(
|
|
{
|
|
user: "user_name",
|
|
pwd: "user_pass",
|
|
roles: [ { role: "readWrite", db: "texts" } ]
|
|
}
|
|
)
|
|
|
|
# Restart mongod.
|
|
|
|
# Also useful
|
|
db.dropUser("username")
|
|
```
|
|
|
|
Then go to `/etc/mongod.conf` and add this:
|
|
Do not use tabs! mongod won't start.
|
|
```bash
|
|
security:
|
|
authorization: enabled
|
|
```
|
|
You need to `use <db>`, `db.auth("username", "pass")` to have access.
|
|
|
|
#### DB migration
|
|
I installed mongo on remote with same user accounts as on local.
|
|
```bash
|
|
# On local.
|
|
$ mongodump --db texts --host mongodb1.example.net --port 3017 --username user --password "pass" --out ./file.db
|
|
|
|
# Rsync it over to remote.
|
|
# On remote.
|
|
# Need to turn off authorization for the drop part.
|
|
|
|
$ mongorestore --db texts --drop --port <port> ./<dbfolder>
|
|
```
|
|
|
|
#### DB collections, needed for web app
|
|
|
|
* v2_senses,
|
|
* v2_senses_map,
|
|
* v2_users,
|
|
* v2_user_tokens
|
|
* sskj
|
|
|
|
|
|
## Web app deployment
|
|
We need to set flask to serve the vuejs frontend:
|
|
```python
|
|
# in ./script/flask_app/app.py
|
|
app = Flask(
|
|
__name__,
|
|
static_folder="./vue/dist/static",
|
|
template_folder="./vue/dist"
|
|
)
|
|
```
|
|
Set the correct CORS options.
|
|
|
|
Might need to reinstall the package. Make sure you're in python virtualenv.
|
|
`$ pip3 install -e .`
|
|
|
|
We also need to tell vuejs client to look for the api on server's address.
|
|
```javascript
|
|
// in script/vue_frontend/src/main.js
|
|
api_addr: "http://<my_server_addr>:5004",
|
|
```
|
|
Compile frontend client:
|
|
```bash
|
|
$ cd ./script/vue_frontend
|
|
$ npm run build
|
|
$ rm ../flask_app/vue -rf
|
|
$ cp ./dist ../flask_app/vue -r
|
|
```
|
|
|
|
After preparing frontend, backend and database, run the command:
|
|
```bash
|
|
$ cd ./script
|
|
$ ./autostart.sh
|
|
```
|
|
|
|
## When fixing things in dev environment:
|
|
```bash
|
|
## Backend
|
|
# git clone everything
|
|
$ cd script
|
|
|
|
# unzip some zipped files
|
|
$ ./sherpa --unpack
|
|
|
|
# virtualenv
|
|
$ source venv/bin/activate
|
|
$ pip3 install -e .
|
|
|
|
# start a mongod instance
|
|
$ sudo systemctl start mongod
|
|
|
|
# start the backend server
|
|
$ ./autostart.sh --debug
|
|
|
|
# you can watch the logs with
|
|
$ tail -f log/main.log
|
|
|
|
|
|
## Frontend
|
|
# set the right config file (path to api)
|
|
$ ./sherpa --frontend_config dev
|
|
$ cd vue_frontend
|
|
$ npm run dev
|
|
|
|
|
|
## Done making changes
|
|
# build frontend for production with the right config files
|
|
$ ./sherpa --build_vue
|
|
|
|
# if you've made changes to static .pickle files
|
|
$ ./sherpa --pack
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|