Arvados 1.4.0 release notes

June 5th, 2019

The Arvados team is pleased to announce the latest release, Arvados 1.4.0. It includes several new features, the most exciting of which is the brand new implementation of the Arvados web UI component, Workbench. The new Workbench is currently still in beta, but we’d love for you to test it as part of your daily work and provide us with feedback.


Other highlights include:

  • New Workbench - React-based single page web app with completely new UI design

  • New Java SDK with much more complete API coverage

  • API Server & traditional Workbench upgraded to Rails 5

  • Support for Prometheus metrics reporting in a number of components

  • Packaging changes for Arvados utilities to improve isolation of dependencies

  • Support for Ubuntu 18.04

  • More components ported to Python 3.x and Python 3.7 support added

  • More than 50 bug fixes


There are a couple of important release notes that you should review before upgrading. Installing the python-arvados-python-client distribution package no longer installs the Arvados Python module (ie arvados) system-wide. If you are depending on its availability, you will need to install arvados-python-client from PyPI. Using virtualenv or similar dependency manager is a good way to isolate dependency requirements of different applications.


As we’ve announced previously 1.4.0 is the last release to support the Jobs API (aka Crunch 1). If you haven’t already migrated to Crunch 2 and the Container API, you should make plans to as soon as possible. Please contact us if you have any questions or require assistance.


You can find a full list of the features and bug fixes on the release page and included below.



#

Subject

12958

[Federation] Workbench login chooser

14291

[crunch-dispatch-cloud] AWS driver

14322

[CWL] Accept collection uuid in input

14324

[crunch-dispatch-cloud] Azure driver

14325

[crunch-dispatch-cloud] Dispatch containers to cloud VMs directly, without slurm or nodemanager

14573

[Spike] [API] Fully functional filename search

14718

[API] Option to issue salted token in login procedure

14720

[Federation] Workbench2 login chooser

14819

Upgrade the arvados/jobs image to Debian Stretch

14853

Merge chapmanb py3k-subprocess32 pull request

14947

[Tests] run-tests.sh --interactive

15064

[Workbench2] Use long-lived cookies to improve login chooser defaults

9945

[SDK] Package Python apps as virtualenvs

13306

arvados-cwl-runner supports Python 3

13823

Package Arvados for Ubuntu Bionic Beaver 18.04 LTS

13937

[keepstore] Move legacy metrics to prometheus /metrics endpoint

13996

[API] Migrate RailsAPI to new cluster config file

14382

Review & merge new contributed Java SDK

14484

[API Server] Return collection size and number of files in collection record

14606

[CWL] CWL v1.1 spec work and support

14645

[arv-mount] Report FUSE operation latency

14669

[Java] Merge into main Arvados repo

14670

[Java] Write / merge Java SDK documentation

14725

[GA4GH] WES 1.0 prototype

14794

Link from Workbench 1 to Workbench 2

14796

[crunch-dispatch-cloud] Document installation / migration from c-d-slurm + node manager

14807

[arvados-dispatch-cloud] Features/fixes needed before first production deploy

14835

Update cluster configuration wiki from scratch doc

14841

Link from Workbench 2 to Workbench 1

14870

[API] Access logs from previous attempts after auto-retrying a container request

14873

[API] Update to Rails 5

14939

Update crunchstat-summary to Python 3

14988

[Workbench] Upgrade to Rails 5

15002

[API] Admin can prevent reuse by cancelling a completed container

15088

[Workbench2] Replicate Workbench1 merge account feature

15180

[Spike] Test migration on production database





#

Subject

8189

[FUSE] Listing a project directory is slow when there are many subprojects

9865

[CWL] Fix undefined behavior after ignoring an unhandled exception

10570

[crunchstat-summary] should not report 0% CPU and 0 RAM in cases where stats were not reported at all

11561

[API] Limit number of lock/unlock cycles for a given container

12026

[crunchstat-summary] gives invalid min_cores recommendation

13593

[API] Sequence of "create group" requests runs slowly, and can crash API server

14326

Our custom-compiled `python-future` and `python3-future` packages can't be installed together and have precedence

14383

[API] Java SDK double slash bug with arvados-controller

14451

crunchstat-summary recommends 563 TB Keep cache (or more!)

14482

[SDKs] Handle "." empty directory placeholder in manifests in Ruby SDK

14539

[SDKs] [arv-mount] Use "." placeholder to persist empty directories

14593

1.3.0 a-c-r broken on crunch1

14627

[crunchstat-summary] job elapsed time wrong for crunch2 jobs

14691

Update suggested .ssh/config section in documentation and workbench

14703

[SSO] Login fails when configured with only the local provider

14723

[CWL] File that should appear multiple times in output collection only appears once

14738

[Workbench] Tag editor not loading

14745

[crunch-dispatch-cloud] Azure cloud driver fixups

14770

Registered workflows fail to run due to spurious --collection-keep-cache=256

14799

[SSO] bump the omniauth-google-oauth2 gem to 0.6.0

14804

[keepstore] Return 5xx (not 4xx) if block is not found due to transient backend device failure

14806

[crunch1] unescape filenames when parsing manifests in crunch-job

14826

[API] Search CentOS certificates path when making callbacks to remote clusters

14828

[crunch-dispatch-cloud] Fix flaky test

14829

Update gem dependencies from Arvados 1.2 to 1.3

14844

[dispatch-cloud] Azure driver bugs discovered in trial run

14845

[controller] Fix flaky test

14880

arv-keepdocker is very picky about the order of its command line arguments and crashes with an obscure error if the order is wrong

14884

arv-put doesn't handle non-ASCII filenames correctly

14885

Review and merge chapmanb's ciso and conda packaging pull request

14900

[arv-put] when reusing a local cache, arv-put does not check if the blocks exist and blindly creates the collection

14911

[dispatch-cloud] Azure driver panic at startup

14920

[crunch-dispatch-cloud] New Azure instances always have state=unknown instead of state=booting

14936

ImportError: No module named future_builtins

14938

[crunch-dispatch-cloud] flaky test in lib/dispatchcloud/worker

14966

[API] Fix hanging test - suspect permission changes

14977

[arvados-dispatch-cloud] kill crunch-run procs for containers that are deleted or have state=Cancelled when dispatcher starts up

15007

[dispatch-cloud] Azure driver panic in Address() (missing nic?)

15040

Fix federation tests

15044

[Tests] Update tests to work on base debian:9 image

15050

Limit number of containers crunch-dispatch-local runs at one time

15058

[SSO] "Not found. Authentication passthru" when using local account authentication

15080

Installing arvados-cwl-runner==1.3.1 from PyPi is broken

15109

[Documentation] Add details about logs table purging on the Admin section

15148

keep-balance incorrectly accounts for blocks in collections with null `modified_at` field

15161

Dev clusters are failing: uninitialized constant ArInternalMetadatum (NameError)

15164

Container request not finalized

15167

Complete logs table management docs with unlogged_attributes setting

15225

[API] installation gives error: Creating git_internal_dir ''...mkdir: cannot create directory ‘’: No such file or directory

15227

[API][Workbench] Adding tags via workbench1's tag editor saves the JSON string to properties

15230

[wb2] Crash loading federated collection

15232

[config] BlobSigningTTL should be expressable with a suffix

15241

[a-c-r] crashing using collection in writable initialWorkDir