frontend_devops fix
This commit is contained in:
@@ -0,0 +1,230 @@
|
||||
# Izdelava vezljivostnih vzorcev za slovenske glagole #
|
||||
|
||||
Študent: Kristjan Voje
|
||||
|
||||
Mentor: prof. Marko Robnik Šikonja
|
||||
Somentorica: dr. Apolonija Gantar
|
||||
|
||||
### Requirements:
|
||||
|
||||
* Linux (built and tested on Ubuntu 16.04),
|
||||
* python3
|
||||
|
||||
## Quick workspace preparation
|
||||
```bash
|
||||
# Clone the repo.
|
||||
$ git clone https://voje@bitbucket.org/voje/diploma.git
|
||||
|
||||
# Prepare the data.
|
||||
$ cd ./data
|
||||
$ unzip data.zip
|
||||
```
|
||||
|
||||
Virtualenv recommended. `$ sudo pip3 install virtualenv`.
|
||||
```bash
|
||||
$ cd ./script
|
||||
$ virtualenv -p /usr/bin/python3 venv
|
||||
|
||||
# A folder venv with python libraries will appear.
|
||||
# To activate the virtual environment:
|
||||
$ source ./venv/bin/activate
|
||||
|
||||
# To deactivate:
|
||||
$ deactivate
|
||||
```
|
||||
|
||||
Now, to build the python packages:
|
||||
```bash
|
||||
# Polyglot dependencies
|
||||
$ sudo apt-get install libicu-dev
|
||||
# NOTE: installing this broke my Arch system.
|
||||
# If on Arch, use the AUR version (it's supposed to be safe).
|
||||
|
||||
$ cd ./script
|
||||
$ pip3 install -e .
|
||||
|
||||
# Polyglot downloads
|
||||
$ polyglot download morph2.sl
|
||||
```
|
||||
|
||||
To test the installation, fire up a python3 shell (while in virtualenv) and:
|
||||
```python
|
||||
import valency
|
||||
```
|
||||
|
||||
## Nodejs environment
|
||||
There's a compiled front end client in the git repo.
|
||||
In case you want to change anything, you'll need to set up the development environment:
|
||||
|
||||
* install nodejs (default Ubuntu versions are usually behind),
|
||||
* check `$ npm -v`, if you don't have it, install npm,
|
||||
```
|
||||
$ npm install vue-cli
|
||||
$ cd ./vue_frontend
|
||||
$ npm install
|
||||
$ npm run dev # for development
|
||||
$ npm run build # for production
|
||||
```
|
||||
You will also need to change some path variables if you'll want to access the backend api from vue development server.
|
||||
See chapter "Web app deployment".
|
||||
|
||||
## MongoDB
|
||||
### Set up the database on the system
|
||||
[install MongoDB on Linux](https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/)
|
||||
|
||||
> The MongoDB instance stores its data files in /var/lib/mongodb and its log files in /var/log/mongodb by default, and runs using the mongodb user account. You can specify alternate log and data file directories in /etc/mongod.conf. See systemLog.path and storage.dbPath for additional information.
|
||||
>
|
||||
> If you change the user that runs the MongoDB process, you must modify the access control rights to the /var/lib/mongodb and /var/log/mongodb directories to give this user access to these directories.
|
||||
|
||||
#### Check if it's working.
|
||||
```bash
|
||||
$ sudo service mongod start / stop / restart
|
||||
$ tail -n 30 /var/log/mongodb/mongod.log
|
||||
|
||||
# If you want the mongo shell on local machine:
|
||||
$ mongo --host 127.0.0.1:<27017> # Check net.port in /etc/mongod.conf
|
||||
```
|
||||
|
||||
#### Security
|
||||
Create admin and user. Localhost only.
|
||||
[sedurity man](https://docs.mongodb.com/v3.4/tutorial/enable-authentication/)
|
||||
Basically, create an admin user then user that admin to create a normal user.
|
||||
Something like this:
|
||||
```
|
||||
use admin
|
||||
|
||||
db.createUser(
|
||||
{
|
||||
user: "admin_name",
|
||||
pwd: "admin_pass",
|
||||
roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
|
||||
}
|
||||
)
|
||||
|
||||
# Restart mongod.
|
||||
|
||||
use texts
|
||||
|
||||
db.createUser(
|
||||
{
|
||||
user: "user_name",
|
||||
pwd: "user_pass",
|
||||
roles: [ { role: "readWrite", db: "texts" } ]
|
||||
}
|
||||
)
|
||||
|
||||
# Restart mongod.
|
||||
|
||||
# Also useful
|
||||
db.dropUser("username")
|
||||
```
|
||||
|
||||
Then go to `/etc/mongod.conf` and add this:
|
||||
Do not use tabs! mongod won't start.
|
||||
```bash
|
||||
security:
|
||||
authorization: enabled
|
||||
```
|
||||
You need to `use <db>`, `db.auth("username", "pass")` to have access.
|
||||
|
||||
#### DB migration
|
||||
I installed mongo on remote with same user accounts as on local.
|
||||
```bash
|
||||
# On local.
|
||||
$ mongodump --db texts --host mongodb1.example.net --port 3017 --username user --password "pass" --out ./file.db
|
||||
|
||||
# Rsync it over to remote.
|
||||
# On remote.
|
||||
# Need to turn off authorization for the drop part.
|
||||
|
||||
$ mongorestore --db texts --drop --port <port> ./<dbfolder>
|
||||
```
|
||||
|
||||
#### DB collections, needed for web app
|
||||
|
||||
* v2_senses,
|
||||
* v2_senses_map,
|
||||
* v2_users,
|
||||
* v2_user_tokens
|
||||
* sskj
|
||||
|
||||
|
||||
## Web app deployment
|
||||
We need to set flask to serve the vuejs frontend:
|
||||
```python
|
||||
# in ./script/flask_app/app.py
|
||||
app = Flask(
|
||||
__name__,
|
||||
static_folder="./vue/dist/static",
|
||||
template_folder="./vue/dist"
|
||||
)
|
||||
```
|
||||
Set the correct CORS options.
|
||||
|
||||
Might need to reinstall the package. Make sure you're in python virtualenv.
|
||||
`$ pip3 install -e .`
|
||||
|
||||
We also need to tell vuejs client to look for the api on server's address.
|
||||
```javascript
|
||||
// in script/vue_frontend/src/main.js
|
||||
api_addr: "http://<my_server_addr>:5004",
|
||||
```
|
||||
Compile frontend client:
|
||||
```bash
|
||||
$ cd ./script/vue_frontend
|
||||
$ npm run build
|
||||
$ rm ../flask_app/vue -rf
|
||||
$ cp ./dist ../flask_app/vue -r
|
||||
```
|
||||
|
||||
After preparing frontend, backend and database, run the command:
|
||||
```bash
|
||||
$ cd ./script
|
||||
$ ./autostart.sh
|
||||
```
|
||||
|
||||
## When fixing things in dev environment:
|
||||
```bash
|
||||
## Backend
|
||||
# git clone everything
|
||||
$ cd script
|
||||
|
||||
# unzip some zipped files
|
||||
$ ./sherpa --unpack
|
||||
|
||||
# virtualenv
|
||||
$ source venv/bin/activate
|
||||
$ pip3 install -e .
|
||||
|
||||
# start a mongod instance
|
||||
$ sudo systemctl start mongod
|
||||
|
||||
# start the backend server
|
||||
$ ./autostart.sh --debug
|
||||
|
||||
# you can watch the logs with
|
||||
$ tail -f log/main.log
|
||||
|
||||
|
||||
## Frontend
|
||||
# set the right config file (path to api)
|
||||
$ ./sherpa --frontend_config dev
|
||||
$ cd vue_frontend
|
||||
$ npm run dev
|
||||
|
||||
|
||||
## Done making changes
|
||||
# build frontend for production with the right config files
|
||||
$ ./sherpa --build_vue
|
||||
|
||||
# if you've made changes to static .pickle files
|
||||
$ ./sherpa --pack
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user