Arvados 1.4.0 Release Notes

June 5th, 2019

The Arvados team is pleased to announce the latest release, Arvados 1.4.0. It includes several new features, the most exciting of which is the brand new implementation of the Arvados web UI component, Workbench. The new Workbench is currently still in beta, but we’d love for you to test it as part of your daily work and provide us with feedback.

Other highlights include:

  • New Workbench - React-based single page web app with completely new UI design
  • New Java SDK with much more complete API coverage
  • API Server & traditional Workbench upgraded to Rails 5
  • Support for Prometheus metrics reporting in a number of components
  • Packaging changes for Arvados utilities to improve isolation of dependencies
  • Support for Ubuntu 18.04
  • More components ported to Python 3.x and Python 3.7 support added
  • More than 50 bug fixes

There are a couple of important release notes that you should review before upgrading. Installing the python-arvados-python-client distribution package no longer installs the Arvados Python module (ie arvados) system-wide. If you are depending on its availability, you will need to install arvados-python-client from PyPI. Using virtualenv or similar dependency manager is a good way to isolate dependency requirements of different applications.

As we’ve announced previously 1.4.0 is the last release to support the Jobs API (aka Crunch 1). If you haven’t already migrated to Crunch 2 and the Container API, you should make plans to as soon as possible. Please contact us if you have any questions or require assistance.

You can find a full list of the features and bug fixes on the release page and included below.

# Subject
12958 [Federation] Workbench login chooser
14291 [crunch-dispatch-cloud] AWS driver
14322 [CWL] Accept collection uuid in input
14324 [crunch-dispatch-cloud] Azure driver
14325 [crunch-dispatch-cloud] Dispatch containers to cloud VMs directly, without slurm or nodemanager
14573 [Spike] [API] Fully functional filename search
14718 [API] Option to issue salted token in login procedure
14720 [Federation] Workbench2 login chooser
14819 Upgrade the arvados/jobs image to Debian Stretch
14853 Merge chapmanb py3k-subprocess32 pull request
14947 [Tests] run-tests.sh --interactive
15064 [Workbench2] Use long-lived cookies to improve login chooser defaults
9945 [SDK] Package Python apps as virtualenvs
13306 arvados-cwl-runner supports Python 3
13823 Package Arvados for Ubuntu Bionic Beaver 18.04 LTS
13937 [keepstore] Move legacy metrics to prometheus /metrics endpoint
13996 [API] Migrate RailsAPI to new cluster config file
14382 Review & merge new contributed Java SDK
14484 [API Server] Return collection size and number of files in collection record
14606 [CWL] CWL v1.1 spec work and support
14645 [arv-mount] Report FUSE operation latency
14669 [Java] Merge into main Arvados repo
14670 [Java] Write / merge Java SDK documentation
14725 [GA4GH] WES 1.0 prototype
14794 Link from Workbench 1 to Workbench 2
14796 [crunch-dispatch-cloud] Document installation / migration from c-d-slurm + node manager
14807 [arvados-dispatch-cloud] Features/fixes needed before first production deploy
14835 Update cluster configuration wiki from scratch doc
14841 Link from Workbench 2 to Workbench 1
14870 [API] Access logs from previous attempts after auto-retrying a container request
14873 [API] Update to Rails 5
14939 Update crunchstat-summary to Python 3
14988 [Workbench] Upgrade to Rails 5
15002 [API] Admin can prevent reuse by cancelling a completed container
15088 [Workbench2] Replicate Workbench1 merge account feature
15180 [Spike] Test migration on production database
# Subject
8189 [FUSE] Listing a project directory is slow when there are many subprojects
9865 [CWL] Fix undefined behavior after ignoring an unhandled exception
10570 [crunchstat-summary] should not report 0% CPU and 0 RAM in cases where stats were not reported at all
11561 [API] Limit number of lock/unlock cycles for a given container
12026 [crunchstat-summary] gives invalid min_cores recommendation
13593 [API] Sequence of "create group" requests runs slowly, and can crash API server
14326 Our custom-compiled `python-future` and `python3-future` packages can't be installed together and have precedence
14383 [API] Java SDK double slash bug with arvados-controller
14451 crunchstat-summary recommends 563 TB Keep cache (or more!)
14482 [SDKs] Handle "." empty directory placeholder in manifests in Ruby SDK
14539 [SDKs] [arv-mount] Use "." placeholder to persist empty directories
14593 1.3.0 a-c-r broken on crunch1
14627 [crunchstat-summary] job elapsed time wrong for crunch2 jobs
14691 Update suggested .ssh/config section in documentation and workbench
14703 [SSO] Login fails when configured with only the local provider
14723 [CWL] File that should appear multiple times in output collection only appears once
14738 [Workbench] Tag editor not loading
14745 [crunch-dispatch-cloud] Azure cloud driver fixups
14770 Registered workflows fail to run due to spurious --collection-keep-cache=256
14799 [SSO] bump the omniauth-google-oauth2 gem to 0.6.0
14804 [keepstore] Return 5xx (not 4xx) if block is not found due to transient backend device failure
14806 [crunch1] unescape filenames when parsing manifests in crunch-job
14826 [API] Search CentOS certificates path when making callbacks to remote clusters
14828 [crunch-dispatch-cloud] Fix flaky test
14829 Update gem dependencies from Arvados 1.2 to 1.3
14844 [dispatch-cloud] Azure driver bugs discovered in trial run
14845 [controller] Fix flaky test
14880 arv-keepdocker is very picky about the order of its command line arguments and crashes with an obscure error if the order is wrong
14884 arv-put doesn't handle non-ASCII filenames correctly
14885 Review and merge chapmanb's ciso and conda packaging pull request
14900 [arv-put] when reusing a local cache, arv-put does not check if the blocks exist and blindly creates the collection
14911 [dispatch-cloud] Azure driver panic at startup
14920 [crunch-dispatch-cloud] New Azure instances always have state=unknown instead of state=booting
14936 ImportError: No module named future_builtins
14938 [crunch-dispatch-cloud] flaky test in lib/dispatchcloud/worker
14966 [API] Fix hanging test - suspect permission changes
14977 [arvados-dispatch-cloud] kill crunch-run procs for containers that are deleted or have state=Cancelled when dispatcher starts up
15007 [dispatch-cloud] Azure driver panic in Address() (missing nic?)
15040 Fix federation tests
15044 [Tests] Update tests to work on base debian:9 image
15050 Limit number of containers crunch-dispatch-local runs at one time
15058 [SSO] "Not found. Authentication passthru" when using local account authentication
15080 Installing arvados-cwl-runner==1.3.1 from PyPi is broken
15109 [Documentation] Add details about logs table purging on the Admin section
15148 keep-balance incorrectly accounts for blocks in collections with null `modified_at` field
15161 Dev clusters are failing: uninitialized constant ArInternalMetadatum (NameError)
15164 Container request not finalized
15167 Complete logs table management docs with unlogged_attributes setting
15225 [API] installation gives error: Creating git_internal_dir ''...mkdir: cannot create directory ‘’: No such file or directory
15227 [API][Workbench] Adding tags via workbench1's tag editor saves the JSON string to properties
15230 [wb2] Crash loading federated collection
15232 [config] BlobSigningTTL should be expressable with a suffix
15241 [a-c-r] crashing using collection in writable initialWorkDir