If you are developing multiple plugins at the same time a "workspace" using Docker and Docker Compose can make the process easier. Creating a workspace this way also generates a PyCharm project.
The basic idea is to use the parent directory of the plugins, so that they can share a common npm installation and docker setup.
The generator contains a special command to setup a workspace based on a Phovea product definition. This will clone all the repositories, download data packages, install dependencies, and build docker for you. By convention, product definition repositories are ending with the _product
suffix.
yo phovea:setup-workspace Caleydo/example_product
Attention!
Replace the product definition (“Caleydo/example_product”) with a real one.
For example, setting up the workspace for TaCo:
yo phovea:setup-workspace taco_product
The first part of the command (yo
) is the Yeoman command-line tool. The second part tells Yeoman that it should use the “phovea” generator and execute “setup-workspace” sub-generator. The third part is the product definition from which the workspace should be created. A prompt will appear asking if the dependencies should be cloned via SSH or HTTPS. Cloning via HTTPS should work everywhere, even behind a proxy, but it requires to authenticate via username and password to access private git repositories / plugins. When cloning via SSH authentication is handled automatically with private and public keys, however when being behind a proxy this method of cloning may not work. See also the documentation on GitHub.
This command will create a new folder, which is named after the product without the _product
suffix (in this case example
). Inside this directory all needed plugins will be cloned. Cloning private repositories requires having the correct access rights and authentication either with SSH or username and password. After cloning all needed repositories their dependencies are collected and merged into the corresponding file inside the root directory of the workspace (for more information see the Folder structure section). The generator then executes the command npm install
to install the node modules. Finally the docker containers will be downloaded and installed. The corresponding command to achieve this is docker-compose build
. Building the containers can fail when being behind a proxy server because Docker does not know anything about the proxy settings. If this is the case, please refer to Running Docker behind a Proxy Server. When the build fails it can be restarted using the following command inside the workspace:
docker-compose build
The workspace can now be opened using PyCharm.
mkdir workspace
cd workspace
git clone https://github.com/phovea/phovea_server.git
# AND/OR clone any other repos that you will need.
yo phovea:resolve
yo phovea:workspace
npm install
docker-compose up
To add other Phovea plugins, halt Docker by hitting ctrl-C, and then:
# For example, if you want to support interactions with a SQL database:
git clone git@github.com:phovea/phovea_data_sql.git
yo phovea:resolve
yo phovea:workspace
npm install
docker-compose build api
docker-compose up
┣━ Root folder (e.g. example)
┃ ┣━ _backup/
┃ ┣━ _data/
┃ ┣━ ordino/
┃ ┣━ phovea_server/
┃ ┣━ phovea_clue/
┃ ┣━ …
┃ ┣━ phovea.js
┃ ┣━ phovea_registry.js
┃ ┣━ package.json
┃ ┣━ requirements.txt
┃ ┣━ docker-compose.yml
┃ ┣━ docker-compose-patch.yml
┃ ┣━ forEach.sh
┃ ┣━ forEach.cmd
┃ ┣━ docker-backup.sh
┃ ┣━ docker-backup.cmd
┃ ┣━ …
The folders inside the workspace root are plugins. An exception to this are folders starting with an underscore (_
), which are not included in the workspace by default. These folders are not plugins but the place to put in database dumps (_backup
) to be restored or other files, which should be served by the web server (_data
). The created workspace contains some plugins by default, which are needed for the specified product. Many of the files (e.g., package.json
, requirements.txt
, docker-compose.yml
, ...) are automatically generated. For example, each plugin maintains its own package.json
file, but when installing the node modules for the workspace a package.json
file, which combines the dependencies of all plugins, is needed. After changing one of these files the workspace needs to be updated (see Updating the workspace) such that the global workspace files include the latest changes. This step is done by default when creating the workspace.
Attention!
The folders beginning with an underscore (_
) are not part of the default workspace.
If a proxy server needs to be configured, change the respective proxy settings inside the docker-compose-patch.yml
file:
...
environment:
http_proxy=yourproxylocation
https_proxy=yourproxylocation
...
After configuring the proxy server the workspace needs to be updated, such that the proxy settings are copied into the docker-compose.yml
file:
yo phovea:update
Then restart the build process:
docker-compose build
Workspace update
./forEach git pull
yo phovea:update
The workspace contains a helper script for both Windows and Linux to restore a database dump. Place the dump under _backup
and execute the following command:
./docker-backup restore
A workspace’s auto-generated files are created using the corresponding files inside each plugin. So for example the package.json
file is created by taking the package.json
file of each plugin and combining them. It is important to update the workspace whenever one of these files (e.g. package.json
, requirements.txt
, ...) changes inside a plugin, because otherwise the changes will not be available in the compiled application.
The workspace contains the forEach.sh
shell script, which is used to update all plugins to the latest version. It will run through all plugins and execute git pull
on the current branch.
./forEach git pull
To update the workspace’s auto-generated files, the Phovea’s Yeoman generator can be used again:
yo phovea:update
This command will check which files have changed and ask if they should be replaced.
To compile and run the application inside a workspace, the Docker containers need to be started first because they contain the backend. The frontend application can be started using the NPM start script.
docker-compose up -d
npm start
The first command will start docker in the background (-d
flag) and the current terminal window is free to execute the NPM start script. The logs from the docker containers can be printed by executing the following command:
docker-compose logs
Alternatively, docker can also be started in the foreground, where it shows the logs instantly. In this case another terminal window has to be used to start the application via the NPM start script.
docker-compose up
Usually, the application runs on localhost:8080, but in case port 8080 is already occupied the server will use the next open port (e.g., 8081). The information on which port the application is running can be seen at the beginning of the output of the start script.
First shut down the NPM start script by clicking on the terminal window that runs the script and press CTRL+C
. If docker-compose is running in the foreground, it can be stopped using the same steps, otherwise it can be stopped using:
docker-compose stop
If the Docker Toolbox is used, the virtual machine needs to be stopped by executing:
docker-machine stop default