How To Create A Workspace
A quick summary of How To Create A Workspace
If you are developing multiple plugins at the same time a "workspace" using Docker and Docker Compose can make the process easier. Creating a workspace this way also generates a PyCharm project.
The basic idea is to use the parent directory of the plugins, so that they can share a common npm installation and docker setup.
Setting up a workspace from exisiting product
The generator contains a special command to setup a workspace based on a Phovea product definition. This will clone all the repositories, download data packages, install dependencies, and build docker for you. By convention, product definition repositories are ending with the
yo phovea:setup-workspace Caleydo/example_product
Replace the product definition (“Caleydo/example_product”) with a real one.
For example, setting up the workspace for TaCo:
yo phovea:setup-workspace taco_product
The first part of the command (
yo) is the Yeoman command-line tool. The second part tells Yeoman that it should use the “phovea” generator and execute “setup-workspace” sub-generator. The third part is the product definition from which the workspace should be created. A prompt will appear asking if the dependencies should be cloned via SSH or HTTPS. Cloning via HTTPS should work everywhere, even behind a proxy, but it requires to authenticate via username and password to access private git repositories / plugins. When cloning via SSH authentication is handled automatically with private and public keys, however when being behind a proxy this method of cloning may not work. See also the documentation on GitHub.
This command will create a new folder, which is named after the product without the
_product suffix (in this case
example). Inside this directory all needed plugins will be cloned. Cloning private repositories requires having the correct access rights and authentication either with SSH or username and password. After cloning all needed repositories their dependencies are collected and merged into the corresponding file inside the root directory of the workspace (for more information see the Folder structure section). The generator then executes the command
npm install to install the node modules. Finally the docker containers will be downloaded and installed. The corresponding command to achieve this is
docker-compose build. Building the containers can fail when being behind a proxy server because Docker does not know anything about the proxy settings. If this is the case, please refer to Running Docker behind a Proxy Server. When the build fails it can be restarted using the following command inside the workspace:
The workspace can now be opened using PyCharm.
Create a new workspace without existing product
- Create a directory where you want to have all the sources:
mkdir workspace cd workspace
- Optional: Clone a repository, you want to change
git clone https://github.com/phovea/phovea_server.git # AND/OR clone any other repos that you will need.
- Resolve phovea dependencies and create workspace
yo phovea:resolve yo phovea:workspace npm install docker-compose up
To add other Phovea plugins, halt Docker by hitting ctrl-C, and then:
# For example, if you want to support interactions with a SQL database: git clone firstname.lastname@example.org:phovea/phovea_data_sql.git yo phovea:resolve yo phovea:workspace npm install docker-compose build api docker-compose up
┣━ Root folder (e.g. example) ┃ ┣━ _backup/ ┃ ┣━ _data/ ┃ ┣━ ordino/ ┃ ┣━ phovea_server/ ┃ ┣━ phovea_clue/ ┃ ┣━ … ┃ ┣━ phovea.js ┃ ┣━ phovea_registry.js ┃ ┣━ package.json ┃ ┣━ requirements.txt ┃ ┣━ docker-compose.yml ┃ ┣━ docker-compose-patch.yml ┃ ┣━ forEach.sh ┃ ┣━ forEach.cmd ┃ ┣━ docker-backup.sh ┃ ┣━ docker-backup.cmd ┃ ┣━ …
The folders inside the workspace root are plugins. An exception to this are folders starting with an underscore (
_), which are not included in the workspace by default. These folders are not plugins but the place to put in database dumps (
_backup) to be restored or other files, which should be served by the web server (
_data). The created workspace contains some plugins by default, which are needed for the specified product. Many of the files (e.g.,
docker-compose.yml, ...) are automatically generated. For example, each plugin maintains its own
package.json file, but when installing the node modules for the workspace a
package.json file, which combines the dependencies of all plugins, is needed. After changing one of these files the workspace needs to be updated (see Updating the workspace) such that the global workspace files include the latest changes. This step is done by default when creating the workspace.
The folders beginning with an underscore (
_) are not part of the default workspace.
Configure Workspace Files
If a proxy server needs to be configured, change the respective proxy settings inside the
... environment: http_proxy=yourproxylocation https_proxy=yourproxylocation ...
After configuring the proxy server the workspace needs to be updated, such that the proxy settings are copied into the
Then restart the build process:
./forEach git pull
Restoring a database dump
The workspace contains a helper script for both Windows and Linux to restore a database dump. Place the dump under
_backup and execute the following command:
Updating the workspace
A workspace’s auto-generated files are created using the corresponding files inside each plugin. So for example the
package.json file is created by taking the
package.json file of each plugin and combining them. It is important to update the workspace whenever one of these files (e.g.
requirements.txt, ...) changes inside a plugin, because otherwise the changes will not be available in the compiled application.
The workspace contains the
forEach.sh shell script, which is used to update all plugins to the latest version. It will run through all plugins and execute
git pull on the current branch.
./forEach git pull
To update the workspace’s auto-generated files, the Phovea’s Yeoman generator can be used again:
This command will check which files have changed and ask if they should be replaced.
Running the application
To compile and run the application inside a workspace, the Docker containers need to be started first because they contain the backend. The frontend application can be started using the NPM start script.
docker-compose up -d npm start
The first command will start docker in the background (
-d flag) and the current terminal window is free to execute the NPM start script. The logs from the docker containers can be printed by executing the following command:
Alternatively, docker can also be started in the foreground, where it shows the logs instantly. In this case another terminal window has to be used to start the application via the NPM start script.
Usually, the application runs on localhost:8080, but in case port 8080 is already occupied the server will use the next open port (e.g., 8081). The information on which port the application is running can be seen at the beginning of the output of the start script.
Shutting the application down
First shut down the NPM start script by clicking on the terminal window that runs the script and press
CTRL+C. If docker-compose is running in the foreground, it can be stopped using the same steps, otherwise it can be stopped using:
If the Docker Toolbox is used, the virtual machine needs to be stopped by executing:
docker-machine stop default