card Arches 5.0 release notes The Arches team has been busy improving Arches and fixing several bugs as well. Below you'll find a listing of more changes included in the latest release. Some of the highlights: - Dependency Upgrades - Arches now uses: - Python 3.7 - Django 2.2 - Elasticsearch 7.4 - Postgres 12 - PostGIS 3 - Task Management: Arches can now be configured to delegate background tasks using Celery - Notifications: Arches can notify users when a task is complete and developers can add custom notifications - Search Export: Users can export search results to CSV or shapefile formats. - Search Components: Developers can now override search components to customize the search interface, or add their own search components - Package Ontologies: Ontologies can now be loaded from a package - The default ontology has been removed to better accommodate instances without ontologies or custom ontologies - Workflows: Developers can design workflows to simplify data management operations for their users - The tile server (TileStache) has been removed in favor of storing vector tiles directly in Postgres - A grouping card component has been added to cards into a single card - A map component has been added to manage multiple geometry nodes in a single map interface. - The map widget has been upgraded with an improved interface - Custom indexes have been added to allow developers to add their own indexes ### Important: The method of creating generic relationships is still available in Arches. However, this feature will be deprecated in a future release. Users are strongly encouraged to use the Resource Instance datatype to relate resources. #### Upgrading Arches 1. Upgrade to Arches 4.4.3 2. Set up a Postgres 12 service with PostGIS 3.0. Ideally this will be on a separate server from the one supporting Arches 4. 3. Carefully follow instructions to upgrade each Arches project database to Postgres 12. https://www.postgresql.org/docs/current/upgrading.html 4. Install Elasticsearch 7.4 5. Install Python 3.6 or 3.7. If you are running Arches on Ubuntu 18.04 we recommend version 3.6 6. Create a Python 3 virtual environment, activate your new virtual environment and install Arches: python -m venv env source env/bin/activate (env) pip install arches 7. Within your project with your Python 3 virtual environment activated: python manage.py es delete_indexes python manage.py es setup_indexes python manage.py migrate python manage.py es index_database 8. (optional) Install a message queue and set up the task manager. https://arches.readthedocs.io/en/latest/task-management/ #### Upgrading an Arches project 1. Make the following changes to your project's package.json file: Add: "@mapbox/geojsonhint": "^3.0.0" "@mapbox/mapbox-gl-geocoder": "^4.4.0" Upgrade: "@mapbox/mapbox-gl-draw": "1.1.2", "bootstrap": "3.3.7", "ckeditor": "^4.6.2", "knockout": "3.5.0", "mapbox-gl": "^1.0.0", "numeral": "^2.0.6", Remove: "mathjs": "^3.11.4", "nvd3": "^1.8.5", and finally run: `yarn install` 2. Because TileStache has been removed from core arches, any services that were provided by the tile server will need to be provided by another server such as GeoServer. ### Known Issues: - The Docker install needs to be updated to support the dependency changes. # Testing Script Before Version Release, go through this checklist to confirm that Arches is running as intended. ## Index | Test Subject | Chrome | Firefox | UI | Notes | | -------------- | :-------------: | :-------------: | ------------------------- | ------------------------------------ | | (Test Subject) | (use indicator from list below) | (use indicator from list below) | ? (to confirm that the UI has rendered correctly) or :x: (to confirm that the UI failed to render correctly) | (add ticket #, details on bug, etc.) | When doing a test pass, consider using these status indicators: :white_check_mark: = Tested & Approved :x: = Merge blocking :construction: = Non-blocking bugs :ok: = Issue has been fixed :question: = Open question * * * ## Install Assigned to: Namjun | Test Subject | Chrome | Firefox | UI | Notes | | -------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Confirm that upgrading from the previous release is issue free | :question: | ? | ? | setup_indexes -> migrate -> index_database | * * * ## Authentication Assigned to: Alexei Ensure that all browsers are compatible with Authentication process. | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------------------------------------------------------ | :----: | :-----: | --- | ----- | | User can sign themselves up for a new Arches account | :question: | ? | ? |#5746 #5814| | User is added to default group (Crowdsource Editor) | :question: | ? | ? | - | | User can log in with their email address | :question: | ? | ? | - | | User can log in with their user_name | :question: | ? | ? | - | | User can reset their password | :question: | ? | ? | - | | User can edit their profile (First and Last name, email address, etc...) | :question: | ? | ? | - | | Admin User can create account with lesser privileges, login as them |:question:| ? | ? | - | | Setting permissions for a user or group as No Access removes the user and group from the permissions list under the permissions tab. |:question:| ? | ? | - | * * * ## System Settings Assigned to: #### Basic Settings | Test Subject | Chrome | Firefox | UI | Notes | | ---------------------------------------------------------------------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Project Name - Updating name updates in index.htm and the page tab |:question:| ? | ? | - | | Web Analytics - String value inserts in base.htm at the location of this template variable:{{GOOGLE_ANALYTICS_TRACKING_ID}} |:question:| ? | ? | - | #### Map Settings Assigned to: Galen | Test Subject | Chrome | Firefox | UI | Notes | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Mapbox API Key - Key saves and API calls are successful |:question:| ? | ? | - | | Hex Grid Precision - Saves properly, but errors if precision is too high (\`Exception detail: TransportError(400, u'parsing_exception', u'[geohash_grid] failed to parse field [precision]')``) |:question:| ? | ? | - | | Hex Cell Size - Changes reflected in Search results |:question:| ? | ? | - | | Default Zoom - Changes reflected in Card Config Manager |:question:| ? | ? | - | | Min Zoom - Changes reflected in Card Config Manager |:question:| ? | ? | - | | Max Zoom - Changes reflected in Card Config Manager |:question:| ? | ? | - | | Project Extent - Changes reflected in Card Config Manager |:question:| ? | ? | - | #### Search Settings Assigned to: Galen Basic Search Settings | Test Subject | Chrome | Firefox | UI | Notes | | -------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Searches per page updates properly in Search |:question:| ? | ? | - | | Number of search suggestions is reflected in search term input |:question:| ? | ? | - | Saved Searches | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------------------------------------------------------------------------------------------------ | :----: | :-----: | --- | ----- | | A new search saves with a name, search url, description, and image and displays properly in the saved search page. |:question:| ? | ? | - | | Users can delete a saved search |:question:| ? | ? | - | * * * ## Map Widget Assigned to: Galen Test in the Card Configuration Manager. | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------------------------------------------------------------------------------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Point line and poly geoms can be created, edited, and deleted |:question:| ? | ? | - | | XY widget is working properly |:question:| ? | ? | - | | Valid geojson entered in the geojson input adds features to the map and pans to those features. If geojson is invalid user has a chance to update data. |:question:| ? | ? | - | | Overlays support custom popups |:construction:| ? | ? |Not supported yet| * * * ## Map Layer Manager Assigned to: Rob #### Resource Layers Assigned to: Rob | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------------------------------------------------------------------------------------------------------------------ | :----: | :-----: | --- | ----- | | Settings - changes to the name and icon of a layer are reflected in the map widget overlay list and overlay library | :question: | ? | ? | - | | Deactivating/Activating a resource layer hides/shows the layer in the map widget overlay list and overlay library | :question: | ? | ? | - | | Style Settings - changes to the layer style are displayed in the layer | :question: | ? | ? | - | | Style Settings Advanced - changes to the layer style are displayed in the layer | :question: | ? | ? | - | #### Clustering (Resource Layers) Assigned to: Rob | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------------------------------------------------------------------------ | :----: | :-----: | --- | ----- | | Increasing cluster distance causes features to cluster at increased distances between them | :question: | ? | ? | - | | Increasing cluster max zoom causes clusters to be formed at higher zoom levels | :question: | ? | ? | - | | Decreasing min points to 2 points causes clusters to form with only 2 points | :question: | ? | ? | - | #### Basemaps Assigned to: Rob | Test Subject | Chrome | Firefox | UI | Notes | | ----------------------------------------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Changing the default search basemap in the basemap settings is reflected on the search page | :question: | ? | ? | - | | Settings - changes to the name and icon of a layer are reflected in the map widget basemap list | :question: | ? | ? | - | | User can delete a basemap and it no longer appears in the map widget's list of basemaps | :question: | ? | ? | - | #### Overlays Assigned to: Rob | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------------------------------------------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Settings - changes to the name and icon of a layer are reflected in the map widget overlay list and overlay library |:question:| ? | ? | - | | User can delete an overlay and it no longer appears in the map widget overlay library |:question:| ? | ? | - | * * * ## Import/Export Assigned to: Galen | Test Subject | Chrome | Firefox | UI | Notes | | -------------------------- | :----: | :-----: | --- | ----- | | Create_mapping_file |:question:| ? | ? | - | | Import business data (cli) |:question:| ? | ? | - | | Export business data (cli) |:question:| ? | ? | - | | Load package (cli) |:question:| ? | ? | - | | Create package (cli) |:question:| ? | ? | - | * * * ## Resource Instance Management Assigned to: Namjun #### Data Types Confirm that the user is able to edit the following data types. Use the Test model to quickly test all ten data types. Note (GeoJson is covered by map widget testing in a different section) | Test Subject | Chrome | Firefox | UI | Notes | | ---------------------- | :----: | :-----: | --- | ----- | | String |:question: | ? | ? | - | | Concepts |:question: | ? | ? | - | | Domains |:question: | ? | ? | - | | Images |:question: | ? | ? | - | | Dates |:question: | ? | ? | - | | Number |:question: | ? | ? | - | | Boolean |:question: | ? | ? | - | | Resource instance type |:question: | ? | ? | - | | Resource instance-list |:question: | ? | ? | - | | Node data type |:question: | ? | ? | - | | File-list data type |:question: | ? | ? | - | | edtf |:question: | ? | ? | - | #### Resource Descriptors Assigned to: Alexei Updating a resource descriptor should be reflected in the following subjects. | Test Subject | Chrome | Firefox | UI | Notes | | --------------------------------------------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Search results | :question: | ? | ? |- | | Form headings | :question: | ? | ? | - | | Report headings | :question: | ? | ? | - | | Map popups | :question: | ? | ? | - | | Related resource d3 graph and listings | :question: | ? | ? | - | * * * #### Provisional Edit Management Assigned to: Galen | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------------------------------------ | :----: | :-----: | --- | ----- | | Provisional users see indication in a widget that their tile edits were submitted |:question:| ? | ? | - | | Resource reviewers are able to identify provisional tiles and can approve/discard provisional edits |:question:| ? | ? | - | * * * ## Related Resources Assigned to: Galen #### Resource Editor | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------------------------------------ | :----: | :-----: | --- | ----- | | User can add a related resource |:question:| ? | ? | - | | User can delete a related resource |:question:| ? | ? | - | | User can change the properties of related resources |:question:| ? | ? | - | | User can switch between table and force directed graph |:question:| ? | ? | - | | User can page through related resources in table |:question:| ? | ? | - | #### Resource Search | Test Subject | Chrome | Firefox | UI | Notes | | -------------------------------------------------------------------------------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Hovering over a link in the force directed graph opens a panel with source and target node info and list each unique relationship type |:question:| ? | ? | - | | Hovering over a node highlights the adjacent links and the corresponding entry in the node list |:question:| ? | ? | - | | Hovering over a node list entry highlights the corresponding node and its adjacent links |:question:| ? | ? | - | | User can switch between table and force directed graph |:question:| ? | ? | - | | Entering text in the search field filters the list of list entries |:question:| ? | ? | - | * * * ## Search Assigned to: Galen | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :----: | :-----: | --- | ----- | | Free text search | :question: | ? | ? | - | | Concept search | :question: | ? | ? | - | | Map filter search | :question: | ? | ? | - | | Time range based search | :question: | ? | ? | - | | Time wheel search | :question: | ? | ? | - | | Advanced search | :question: | ? | ? | - | | Related resource table search | :question: | ? | ? |Table filter is not available| | Related resource graph search | :question: | ? | ? | - | | Resource type search | :question: | ? | ? | - | | Edit status search (provisional, authoritative, or both). Confirm that only resource reviewers are able to see provisional tile data | :question: | ? | ? | - | | Map should not zoom to points that a user is not permitted to read, nor should the search aggregation layer (e.g. hexbin or heatmap) indicate that a restricted resource is present. | :question: | ? | ? | - | Advanced Search | Test Subject | Chrome | Firefox | UI | Notes | | -------------------------------------------------------------- | :----: | :-----: | --- | ----- | | "Like" and "Not Like" return expected results | :question: | ? | ? | - | | "Equals" and "Not" return expected results | :question: | ? | ? | - | | =,<=,>=,<,> for number-type data return expected results | :question: | ? | ? | - | | Removing or Adding filters updates results as expected | :question: | ? | ? | - | | Multiple filters return expected results | :question: | ? | ? | - | Search Export | Test Subject | Chrome | Firefox | UI | Notes | | -------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Search Export works for nodes of *all* datatypes | :question: | ? | ? | ? | | Search Export downloads immediately if under limit threshold | :question: | ? | ? | ? | | Search Export creates notification if over limit threshold | :question: | ? | ? | ? | | Export disabled if celery not running and over limit threshold | :question: | ? | ? | ? | * * * ## Graph Design Assigned to: Galen ### Arches Designer | Test Subject | Chrome | Firefox | UI | Notes | | ---------------- | :----: | :-----: | --- | ----- | | Import graph |:question:| ? | ? | - | | Export graph |:question:| ? | ? | - | | Clone graph |:question:| ? | ? | - | | Delete graph |:question:| ? | ? | - | | Create branch |:question:| ? | ? | - | | Create graph |:question:| ? | ? | - | | Delete Instances |:question:| ? | ? | - | ### Graph Designer | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------ | :----: | :-----: | --- | ----- | | Import graph |:question:| ? | ? |(?)Popup blocker will block the new windows for some of these functions.| | Export graph |:question:| ? | ? | - | | Clone graph |:question:| ? | ? | - | | Delete graph |:question:| ? | ? | - | | Create branch |:question:| ? | ? | - | | Create graph |:question:| ? | ? | - | | Delete Instances |:question:| ? | ? | - | | Add/Edit cards |:question:| ? | ? | - | | Reorder widgets in card |:question:| ? | ? | - | ### Card Components | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------ | :----: | :-----: | --- | ----- | | Grouping Card works as expected |:question:| ? | ? | - | | Map Card works as expectd |:question:| ? | ? | - | | Related Resources Map Card works as expectd |:question:| ? | ? | - | * * * ## Permissions Management Assigned to: Galen | Test Subject | Chrome | Firefox | UI | Notes | | ------------------------------------------------------------------------------------------------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Confirm removing `read` permissions removes that section from the report |:question:| ? | ? | - | | Confirm removing `read` permissions removes that section from the form |:question:| ? | ? | - | | Confirm removing `read` permissions removes that section from the adv. search |:question:| ? | ? | - | | Confirm removing `read` permissions for a spatial nodegroup removes that resource type from map based search results |:question:| ? | ? | - | | Confirm removing `read` permissions for a spatial nodegroup removes that resource type from the overlays section of the map settings |:question:| ? | ? | - | | Confirm removing `read` permissions for a spatial nodegroup removes the map from the Map Report |:question:| ? | ? | - | | Confirm removing `read` permissions for a date based nodegroup removes the related entries from the type dropdown in the time filter of the search page |:question:| ? | ? | - | | Confirm removing `read` permissions for a date based nodegroup reduces the wheel count appropriately |:question:| ? | ? | - | | Confirm removing `write` permissions but still having read permissions disallows saving that section of the form |:question:| ? | ? | - | * * * ## Reports Assigned to: Galen #### Headers Rendering | Test Subject | Chrome | Firefox | UI | Notes | | ----------------------------------------------------------------------- | :----: | :-----: | --- | ----- | | Confirm that report templates with map header gets rendered correctly |:question:| ? | ? | - | | Confirm that report templates with image header gets rendered correctly |:question:| ? | ? | - | | Confirm that report templates with no header gets rendered correctly |:question:| ? | ? | - | #### Data Rendering | Test Subject | Chrome | Firefox | UI | Notes | | ---------------------- | :----: | :-----: | --- | ----- | | String |:question:| ? | ? | - | | Concepts |:question:| ? | ? | - | | Domains |:question:| ? | ? | - | | Images |:question:| ? | ? | - | | Dates |:question:| ? | ? | - | | Number |:question:| ? | ? | - | | Boolean |:question:| ? | ? | - | | Resource instance type |:question:| ? | ? | - | | Node data type |:question:| ? | ? | - | | File-list type |:question:| ? | ? | - | * * * ## RDM Assigned to: Namjun #### Thesauri | Test Subject | Chrome | Firefox | UI | Notes | | ------------------ | :----: | :-----: | --- | ----- | | Add scheme |:question:| ? | ? |Dialogue contents do not update right away| | Delete scheme |:question:| ? | ? | - | | Import scheme |:question:| ? | ? |Does not import images| | Export scheme |:question:| ? | ? |Does not import images| | Add top concept |:question:| ? | ? | - | | Import from SPARQL |:question:| ? | ? |The tree on the left is not refreshed on import | | Manage parents |:question:| ? | ? |When polyhierachy is set, the selection not deactivated | | Make collection |:question:| ? | ? |If a user creates two collections with the same name they will not be able to delete either of them| | Add label |:question:| ? | ? | - | | Add Note |:question:| ? | ? | - | | Add image |:question:| ? | ? | - | #### Collections | Test Subject | Chrome | Firefox | UI | Notes | | -------------------------------------- | :----: | :-----: | --- | ----- | | Add collection |:question:| ? | ? | - | | Delete collection |:question:| ? | ? | - | | Export all collections |:question:| ? | ? |this action redirects to this url with inline xml file:/concepts/export/collections| | Add dropdown entry |:question:| ? | ? | - | | Add sort order and confirm in dropdown |:question:| ? | ? | - | * * * ## Arches Collector Manager | Test Subject | Chrome | Firefox | UI | Notes | | -------------------------------------------------------------------------------------| :----: | :-----: | --- | ----- | | Create new Project |:question:| ? | ? | - | | Save project upon selecting resource type and data entry card(s) |:question:| ? | ? | - | * * *