QA-tests: update some details

Signed-off-by: Simon L. <szaimen@e.mail.de>
Co-Authored-By: szaimen <42591237+szaimen@users.noreply.github.com>

tests/QA/001-initial-setup.md

@@ -1,10 +1,10 @@
 # Initial setup
 
 - [ ] Verify that after starting the test container, you can access the AIO interface using https://internal.ip.address:8080
-- [ ] After clicking the self-signed-certificate warning away, it should show the setup page with an explanation what AIO is and the initial password and a button that contains a link to the AIO login page
-- [ ] After copying the password and clicking on this button, it should open a new tab with the login page
-- [ ] The login page should show an input field that allows to enter the AIO password and a `Log in` button
-- [ ] After pasting the new password into the input field and clicking on this button button, you should be logged in
+- [ ] After clicking the self-signed-certificate warning away, it should show the setup page with an explanation what AIO is and the initial passphrase and a button that contains a link to the AIO login page
+- [ ] After copying the passphrase and clicking on this button, it should open a new tab with the login page
+- [ ] The login page should show an input field that allows to enter the AIO passphrase and a `Log in` button
+- [ ] After pasting the passphrase into the input field and clicking on this button, you should be logged in
 - [ ] You should now see the containers page and you should see three sections: one general section which explains what AIO is, one `New AIO instance` section and one section that allows to restore the whole AIO instance from backup.
 
 You can now continue with [002-new-instance.md](./002-new-instance.md) or [010-restore-instance.md](./010-restore-instance.md).

tests/QA/002-new-instance.md

@@ -11,7 +11,7 @@ For the below to work, it is important that you have a domain that you point ont
     - [ ] Entering the domain that does point to your server e.g. `yourdomain.com` should finally redirect you to the next screen (if you did not configure your domain yet or did not open port 443, it should report that to you)
 - [ ] Now you should see a button `Start containers` and an explanation which points out that clicking on the button will start the containers and that this can take a long time.
 - [ ] Below that you should see a section `Optional addons` which shows a checkbox list with addons that can be enabled or disabled.
-    - [ ] Collabora and Nextcloud Talk should be enabled, the rest disabled
+    - [ ] Collabora, Imaginary, Talk and Whiteboard should be enabled, the rest disabled
     - [ ] Unchecking/Checking any of these should insert a button that allows to save the set config
     - [ ] Checking OnlyOffice and Collabora at the same time should show a warning that this is not supported and should  not saving the new config
     - [ ] Recommended is to uncheck all options now

tests/QA/004-initial-backup.md

@@ -1,18 +1,32 @@
 # Initial backup
 
-- [ ] In the Backup and restore section, you should now see and input box where you should type in the path where the backup should get created and some explanation below
+- [ ] In the Backup and restore section, you should now see two input boxes where for one you should type in the path where the backup should get created and some explanation below or the other type in a remote ssh location
+- [ ] First, check a local backup:
     - [ ] Enter `/` which should send an error
     - [ ] Enter `/mnt/` or  `/media/` or `/host_mnt/` or `/var/backups/` should send an error as well
     - [ ] Accepted should be `/mnt/backup`, `/media/backup`, `/host_mnt/c/backup` and `/var/backups`.
     - [ ] The side should now reload
+    - [ ] In the Backup restore section you should now see a Backup information section with important info like the encryption password, the backup location and more.
+    - [ ] Also you should see a Backup creation section that contains a `Create backup` button.
+    - [ ] Clicking on the `Create backup` button should open a window prompt that allows to cancel the operation.
+    - [ ] Canceling should return to the website, confirming should reveal the big spinner again which should block the website again.
+    - [ ] After a while you should see the information that Backup container is currently running
+- [ ] another option are remote backups via SSH using borgbackup. The remote borg repo URL must contain both `@` and `:`. The process works as follows:
+    1. You enter a remote borg repo URL (e.g. `ssh://user@host:port/path/to/repo` or `user@host:/path/to/repo`).
+    2. On the first connection attempt, a SSH key pair is generated automatically and the public key is displayed.
+    3. You add the public key to the `~/.ssh/authorized_keys` file on the remote server so that AIO can connect to it.
+    4. Once authorized, AIO can create and restore backups on the remote server.
+    - [ ] Enter `user` (no `@` and no `:`) which should send an error
+    - [ ] Enter `user@host` (no `:`) which should send an error
+    - [ ] Enter `userhost:/path` (no `@`) which should send an error
+    - [ ] Accepted should be `ssh://user@host:22/path/to/repo` or `user@host:/path/to/repo`
+    - [ ] Both a local backup location and a remote repo URL should not be accepted at the same time
+    - [ ] The page should now reload
+    - [ ] Now click on `Create backup`
+    - [ ] After the first failed backup attempt with a remote repo, the SSH public key for borg should be shown so it can be authorized on the remote server
+    - [ ] After authorizing the server on the remote, scroll down and click on `Create backup` again to create another backup. This time it should succeed.
 - [ ] The initial Nextcloud credentials on top of the page that are visible when the containers are running should now be hidden in a details tag
-- [ ] In the Backup restore section you should now see a Backup information section with important info like the encryption password, the backup location and more.
-- [ ] Also you should see a Backup cretion section that contains a `Create backup` button.
-- [ ] Clicking on the `Create backup` button should open a window prompt that allows to cancel the operation.
-- [ ] Canceling should return to the website, confirming should reveal the big spinner again which should block the website again.
-- [ ] After a while you should see the information that Backup container is currently running
-- [ ] Below the Containers section you should see the option to `Start containers` again.
 - [ ] After a while and a few automatic reloads (as long as the side is focused), you should be redirected to the usual page and seen in the Backup and restore section that the last backup was successful.
-- [ ] Below thhat you should see a details tag that allows to reveal all backup options
+- [ ] Below that you should see a details tag that allows to reveal all backup options
 
 You can now continue with [020-backup-and-restore.md](.//020-backup-and-restore.md)

tests/QA/010-restore-instance.md

@@ -2,17 +2,34 @@
 
 For the below to work, you need a backup archive of an AIO instance and the location on the test machine and the password for the backup archive. You can get one here: [backup-archive](./assets/backup-archive/)
 
-- [ ] The section that allows to restore the whole AIO instance from backup should show two input fields: one that allows to enter a location where the backup archive is located and one that allows to enter password of the archive. It should also show a short explanation regarding the path requirements
-- [ ] Entering an incorrect path and/or password should let you continue and test your settings in the next step
-    - [ ] Clicking on the test button should after a reload bring you back to the initial screen where it should say that the test was unsuccessful. Also you should be able to have a look at the backup container logs for investigation what exactly failed.
-    - [ ] You should also now see the input boxes again where you can change the path and password, confirm it and bring you again to the screen where you can test your settings.
-- [ ] Entering the correct path to the backup archive and the correct password here should:
-    - [ ] Should reload and should hide all options except the option to test the path and password
-    - [ ] After the test you should see the options to check the integrity of the backup and a list of backup archives that you can choose from to restore your instance
-    - [ ] Clicking on either option should show a window prompt that lets you cancel the operation
-    - [ ] Clicking on the integrity check option should check the integrity and report that the backup integrity is good after a while which should then only show the option to choose the backup archive that should be restored
-    - [ ] Choosing the restore option should finally restore your files. 
-    - [ ] After waiting a while it should reload the page and should show the usual container interface again with the state of your containers (stopped) and the option to start and update the containers again.
+- [ ] The section that allows to restore the whole AIO instance from backup should show three input fields: one that allows to enter a location where the backup archive is located and one that allows to enter a remote ssh path and one that allows to enter password of the archive. It should also show a short explanation regarding the path requirements
+- [ ] First, check restoring from a local backup location:
+    - [ ] Entering an incorrect path and/or password should let you continue and test your settings in the next step
+        - [ ] Clicking on the test button should after a reload bring you back to the initial screen where it should say that the test was unsuccessful. Also you should be able to have a look at the backup container logs for investigation what exactly failed.
+        - [ ] You should also now see the input boxes again where you can change the path and password, confirm it and bring you again to the screen where you can test your settings.
+    - [ ] Entering the correct path to the backup archive and the correct password here should:
+        - [ ] Should reload and should hide all options except the option to test the path and password
+        - [ ] After the test you should see the options to check the integrity of the backup and a list of backup archives that you can choose from to restore your instance
+        - [ ] Clicking on either option should show a window prompt that lets you cancel the operation
+        - [ ] Clicking on the integrity check option should check the integrity and report that the backup integrity is good after a while which should then only show the option to choose the backup archive that should be restored
+        - [ ] Choosing the restore option should finally restore your files.
+        - [ ] After waiting a while it should reload the page and should show the usual container interface again with the state of your containers (stopped) and the option to start and update the containers again.
+- [ ] Next, check restoring from a remote backup location via SSH. The remote borg repo URL must contain both `@` and `:`. The restore process works as follows:
+    1. You enter a remote borg repo URL (e.g. `ssh://user@host:port/path/to/repo` or `user@host:/path/to/repo`) and the backup password.
+    2. On the first connection attempt, a SSH key pair is generated automatically and the public key is displayed.
+    3. You add the public key to the `~/.ssh/authorized_keys` file on the remote server so that AIO can connect to it.
+    4. Once authorized, AIO can list and restore backups from the remote server.
+    - [ ] Enter an invalid remote repo URL (e.g. `user` without `@` and `:`) which should send an error
+    - [ ] Enter a valid remote borg repo URL and the correct backup password:
+        - [ ] Should reload and should hide all options except the option to test the path and password
+        - [ ] After the first failed connection attempt, the SSH public key for borg should be shown so it can be authorized on the remote server
+        - [ ] After authorizing the key on the remote server, scroll down and click on the test button again. This time it should succeed and show the options to check the integrity and list backup archives
+        - [ ] After the test you should see the options to check the integrity of the backup and a list of backup archives that you can choose from to restore your instance
+        - [ ] Clicking on either option should show a window prompt that lets you cancel the operation
+        - [ ] Clicking on the integrity check option should check the integrity and report that the backup integrity is good after a while which should then only show the option to choose the backup archive that should be restored
+        - [ ] Choosing the restore option should finally restore your files.
+        - [ ] After waiting a while it should reload the page and should show the usual container interface again with the state of your containers (stopped) and the option to start and update the containers again.
+
 - [ ] Clicking on `Start and update containers` should show a window prompt that you should create a backup. Canceling should cancel the operation, confirming should reveal the big spinner again.
 - [ ] After waiting a bit, all containers should be green and your instance should be fully functional again
 

tests/QA/020-backup-and-restore.md

@@ -1,6 +1,6 @@
 # Backup and restore
 
-- [ ] Expanding all backup options in the Backup and restore sectioin should reveal a Backup information section, Backup creation section, Backup check section, Backup restore section and a Daily backup section.
+- [ ] Expanding all backup options in the Backup and restore sectioin should reveal a Backup information section, Backup creation section, Backup check section, Backup restore section and a Daily backup section as well as a additional backup location section
 - [ ] The backup restore section should list all available backup archives and list them from most recent to least recent.
 - [ ] Clicking on either option of Create backup, Check backup integrity or Restore selected backup should run the corresponding action and report after a while in the last check, backup or restore was successful.
 - [ ] Daily backup creatio should allow to enter a time in 24h format e.g. `04:00` should be accepted, `24:00` or `dfjlk` not.

tests/QA/030-aio-password-change.md

@@ -1,12 +1,12 @@
-# AIO password change
+# AIO passphrase change
 
-- [ ] In the AIO password change section you should see two input fields. And below the requirements for a new password
-- [ ] When entering nothing it should report that you need to enter your current aio password
-- [ ] When entering a false password, it should report that to you
-- [ ] After entering your current password and leaving the new password empty it should report that you need to enter a new password
-- [ ] After entering a new passwort shorter than 24 characters or not allowed characters, it should report that the password requirements are not met.
+- [ ] In the AIO passphrase change section you should see two input fields. And below the requirements for a new passphrase
+- [ ] When entering nothing it should report that you need to enter your current AIO passphrase
+- [ ] When entering a false passphrase, it should report that to you
+- [ ] After entering your current passphrase and leaving the new passphrase empty it should report that you need to enter a new passphrase
+- [ ] After entering a new passphrase shorter than 24 characters or not allowed characters, it should report that the passphrase requirements are not met.
 - [ ] `sdfjlksj` should not be accepted
 - [ ] `jdsfklöjiroewoäsadjkfölk` should not be accepted
-- [ ] `sdjlfj SDJFLK 32489 sdjklf` should which should reload the page
+- [ ] `sdjlfj SDJFLK 32489 sdjklf` should be accepted, which should reload the page
 
 You can now continue with [040-login-behavior.md](./040-login-behavior.md)

tests/QA/040-login-behavior.md

@@ -1,7 +1,7 @@
 # Login behavior
 
 - [ ] When opening the AIO interface in a new tab while the apache container is running, it should report on the login page that Nextcloud is running and you should use the automatic login
-- [ ] When the apache container is stopped, you should see here an input field that allows you to enter the AIO password which should log you in
+- [ ] When the apache container is stopped, you should see here an input field that allows you to enter the AIO passphrase which should log you in
 - [ ] Starting and stopping the containers multiple times should every time produce a new token that is used in the admin overview in Nextcloud as link in the button to log you into the AIO interface. (see [003-automatic-login.md](./003-automatic-login.md))
 
 You can now continue with [050-optional-addons.md](./050-optional-addons.md)

tests/QA/050-optional-addons.md

@@ -10,6 +10,8 @@
     - [ ] Imaginary by having a look if when uploading a new picture in Nextcloud, it adds some log entries to the container
     - [ ] Fulltextsearch by trying to search for a heading inside a file in Nextcloud
     - [ ] Talk-recording by starting a call and trying to record something
-- [ ] When Collabora is enabled, it should show below the Optional Addons section a section where you can change the dictionaries for collabora. `de_DE en_GB en_US es_ES fr_FR it nl pt_BR pt_PT ru` should be a valid setting. E.g. `de.De` not. If already set, it should show a button that allows to remove the setting again.
+- [ ] When Collabora is enabled
+    - [ ] It should show below the Optional Addons section a section where you can change the dictionaries for collabora. `de_DE en_GB en_US es_ES fr_FR it nl pt_BR pt_PT ru` should be a valid setting. E.g. `de.De` not. If already set, it should show a button that allows to remove the setting again.
+    - [ ] Also, you should see an input field that allows to enter additional collabora options. E.g. `net.content_security_policy=false` should not be accepted, but `--o:net.content_security_policy="frame-ancestors *.example.com:*;"` should.
 
-You can now continue with [060-environmental-variables.md](./060-environmental-variables.md)
+You can now continue with [055-community-containers.md](./055-community-containers.md)

tests/QA/055-community-containers.md

@@ -0,0 +1,13 @@
+# Community Containers
+
+- [ ] At the very bottom of the page, there should be a Community Containers section
+- [ ] The section should show a details element that allows to reveal the list of available community containers
+- [ ] When containers are running, the checkboxes should be disabled and a notice should inform the user that changes can only be made when containers are stopped
+- [ ] When containers are stopped, checkboxes should be enabled
+    - [ ] Enabling a community container and clicking `Save changes` should show a confirmation dialog
+    - [ ] Canceling the confirmation dialog should not save the changes
+    - [ ] Confirming should save the changes and reload the page
+    - [ ] After saving, the enabled community container should appear in the containers section and start along with the other containers when `Start containers` is clicked
+    - [ ] Disabling a previously enabled community container and saving should remove it from the containers section after stopping and starting containers
+
+You can now continue with [060-environmental-variables.md](./060-environmental-variables.md)

tests/QA/060-environmental-variables.md

@@ -6,6 +6,7 @@
 - [ ] When starting the mastercontainer with `--env TALK_PORT=3479` on a clean instance, the talk container should use this port later on. Using a value here that is not a port will not allow the mastercontainer to start correctly. Also it should stop if apache_port and talk_port are set to the same value.
 - [ ] Make also sure that reverse proxies work by following https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md#reverse-proxy-documentation and following [001-initial-setup.md](./001-initial-setup.md) and [002-new-instance.md](./002-new-instance.md)
 - [ ] When starting the mastercontainer with `--env SKIP_DOMAIN_VALIDATION=true` on a clean instance, it should skip the domain verification. So it should accept any domain that you type in then.
+- [ ] When starting the mastercontainer with `--env DOCKER_API_VERSION=1.44` it should use the mentioned docker API version internally for all requests
 - [ ] When starting the mastercontainer with `--env NEXTCLOUD_DATADIR="/mnt/testdata"` it should map that location from `/mnt/testdata` to `/mnt/ncdata` inside the Nextcloud container. Not having adjusted the permissions correctly before starting the Nextcloud container the first time will not allow the Nextcloud container to start correctly. See https://github.com/nextcloud/all-in-one#how-to-change-the-default-location-of-nextclouds-datadir for allowed values.
 - [ ] When starting the mastercontainer with `--env NEXTCLOUD_MOUNT="/mnt/"` it should map `/mnt/` to `/mnt/` inside the Nextcloud container. See https://github.com/nextcloud/all-in-one#how-to-allow-the-nextcloud-container-to-access-directories-on-the-host for allowed values.
 - [ ] When starting the mastercontainer with `--env NEXTCLOUD_UPLOAD_LIMIT=11G` it should change Nextclouds upload limit to 11G. See https://github.com/nextcloud/all-in-one#how-to-adjust-the-upload-limit-for-nextcloud for allowed values.
@@ -24,6 +25,5 @@ See https://github.com/nextcloud/all-in-one#how-to-trust-user-defined-certificat
 - [ ] When starting the mastercontainer with `--env NEXTCLOUD_ENABLE_DRI_DEVICE=true`, the resulting Nextcloud container should have the /dev/dri device mounted into the container. (Only works if a `/dev/dri` device is present on the host)
 - [ ] When starting the mastercontainer with `--env NEXTCLOUD_ENABLE_NVIDIA_GPU=true`, the resulting Nextcloud container should have the nvidia gpu device mounted into the container. (Only works if a Nvidia GPU and runtime is installed on the host)
 - [ ] When starting the mastercontainer with `--env NEXTCLOUD_KEEP_DISABLED_APPS=true` it should keep apps in Nextcloud that are disabled in the AIO interface. For example if Collabora is disabled in the AIO interface and you install the richdocuments app in Nextcloud, a restart should not uninstall the richdocuments app in Nextcloud anymore.
-- [ ] When starting the mastercontainer with `--env AIO_COMMUNITY_CONTAINERS="fail2ban"`, it should add the fail2ban container to the container stack and show it in the AIO interface as well as start it, etc.
 
 You can now continue with [070-timezone-change.md](./070-timezone-change.md)