Changelog

Here you can see the full list of changes between each ufo-core release.

Version 0.17

Fixes:

  • #163: outline installation with meson

  • #162: check for Python 3 dev files as well

  • #160: build Sphinx manual with meson

Pull requests:

  • #189 from llohse/python-meson

  • #183 from ufo-kit/tests-optional

  • #185 from llohse/mesonpython2

  • #184 from llohse/newsfix

  • #169 from ufo-kit/allow_more_input_nodes

  • #178 from ufo-kit/docker

  • #179 from ufo-kit/channel_order

  • #176 from ufo-kit/abort

  • #175 from ufo-kit/non-dash-task-names

  • #174 from ufo-kit/config

  • #173 from ufo-kit/python3-remains

  • #172 from ufo-kit/fix-opencl-version-warning

  • #171 from ufo-kit/python3

  • #170 from ufo-kit/propagate-expand-errors

  • #166 from ufo-kit/add-12bit-support

Version 0.16

Enhancements:

  • OutputTask signals end via “inputs-stopped” signal

  • InputTask: Improve performance

  • Add ufo_buffer_set/get_layout and pass through

Fixes:

  • Better error propagation

  • Fix #116: allow _get_requisition to report errors

  • Fix #130: add ufo_buffer_swap_data interface

  • Fix #143: invalid object dereference

  • Fix #145: remove sphinx-tabs dependency

  • Fix #146: multi GPU expansion broke

  • Fix #147: do correct comparison

  • Fix #148: re-write longest path algorithm

  • Fix #154: add UFO_RESOURCES_CHECK_SET_AND_RETURN

Breaks:

  • Remove _with_opts functions

  • Retire ufo_graph_{{shallow_}copy|subgraph,flatten}

  • Raise GLib version to 2.38

  • Remove address options from CLI tools

  • Remove daemon and messenger API

Version 0.15.2

This is another bugfix release to help Debian adoption.

Version 0.15.1

This is bugfix release that fixes #143 which causes segfaults with certain task graph conditions.

Version 0.15

Released on January 4th 2018.

Enhancements:

  • Manual and API documentation has been much improved

  • Passing -Dintrospection=false to meson disables introspection data

  • An error is generated if duplicate node names exist in JSON

  • Added ufo_profiler_call_blocking to wait for execution

  • Added ufo_resources_get_kernel_from_source_with_opts

  • Added an NVIDIA Dockerfile example

  • The deadlock warning has been turned into a debug message

  • The scheduler returns early if no OpenCL platform is found

  • Fix #135: added UFO_GPU_NODE_INFO_NAME

Fixes:

  • UFO is compiled with gnu99 now to avoid GLOB_TILDE complaints

  • Fixed wrong references and highlighting in the manual

  • if UFO_DEVICE_TYPE=cpu is set, no GPUs are used

  • Fix #142: fixed multi input and GPU processing

  • Fix #142: prune graph correctly for multiple GPU

  • Fix #137: support JSON array properties

  • Fix #133: output build flag string for each device

  • Fix #132: do not segfault on unknown properties

  • Fix #126: abort execution without resources

  • Fix #129: allocate enough memory for index array

  • Fix #125: build and install manpages with meson

Breaks:

  • Property sets have been removed from the JSON specification

Version 0.14

Released on September 5th 2017.

Enhancements:

  • meson build system support has been added

  • Improve documentation

  • Improve expansion for multi-input node graphs

  • Output warning in case expansion is not possible

  • Use g_printerr to print errors on stderr instead of stdout

  • ufo-launch returns exit code 1 on execution error

  • Make device usage debug output unambiguous

  • #123: allow vector properties via ufo-launch

  • #120: generate buffer timestamps with the –timestamps option

Fixes:

  • Fix a minor memory leak in ufo-launch

  • Fix problem with CMake install dir

  • Fix #119: compile OpenCL programs for each device with device-specific flags to allow kernel developers optimizations for each device

  • Fix #121: use the same command-line interface for both runjson and launch

Breaks:

  • We dropped support for Autotools, CMake and meson are the only options to build at the moment.

Version 0.13

Released on January 25th 2017.

Enhancements:

  • ufo-runjson: Add the -s/–scheduler flag to choose a scheduler different from the regular one.

  • Restructure docs and add section about broadcasts

  • Added ufo_resources_get_kernel_source function to have an API to access directly a source file from CL/kernel path.

Fixes:

  • Fixup for compilation and installation on MacOS

Version 0.12.1

Bugfix release released on November 28th 2016.

  • Do not install the Docbook XML build dir

  • Document UFO_BUFFER_DEPTH_INVALID

  • Enable Large File Support

  • Check ftell and return NULL on error

  • Add forgotten manpage to the build list

  • Initialize uninitialized variable

  • Enable _FORTIFY_SOURCE feature flag

Version 0.12

Released on November 24th 2016.

Enhancements:

  • ufo-launch: convert string to enum values

  • ufo-runjson: add -t/–trace flag doing the same thing as ufo-launch

  • Documentation updates

Fixes:

  • Fix manual heading

  • ufo-mkfilter: fix template and type handling

  • Add manpage for ufo-prof

Breaks:

  • Add UFO_BUFFER_DEPTH_INVALID with value zero which means adding to the API and breaking ABI

  • Remove package target from the build system

Version 0.11.1

Bugfix release released on November 12th 2016.

  • Install systemd unit file through pkg-config

  • Fix #110: install templates correctly

  • Do not run xmllint on manpage generation output

  • Remove unused CMake modules

  • Remove executable bit from source files

  • Remove PACKAGE_* variables

Version 0.11

Released on November 8th 2016

Enhancements:

  • Build manpages for the tools

  • Update TomoPy integration

  • Add UFO_DEVICE_TYPE environment variable

  • Properly build on MacOS

  • Unify debug message output format

  • ufo-launch: rewrite specification parser allowing more flexible descriptions

  • ufo-launch: add –quieter

  • ufo-mkfilter: add –type and –use-gpu

Fixes:

  • Check if we have with multiple roots to exit early

  • Show version information consistently

  • ufo-prof: support Python < 3.0

Breaks:

  • GNUInstallDirs instead is now used instead of our own ConfigurePaths CMake module which might affect installation paths on your system. Please note that for example -DPREFIX thus becomes -DCMAKE_INSTALL_PREFIX.

  • Remove CPack

  • ufod: unused –paths option removed

  • ufo-runjson: unused –path option removed

  • ufo-launch: do not execute graph when –dump’ing

Version 0.10

Released on May 24th 2016

Enhancements:

  • Add UFO_GPU_NODE_INFO_MAX_MEM_ALLOC_SIZE

  • Fix #103: allow ufo-launch to use arbitrary graphs with new workflow specification and more robust parsing

  • Look for AMD APP SDK 3.0

  • Cache programs to avoid rebuilding them

  • ufo-runjson now also outputs the number of processed items

  • Fix #104: output type and blurb with -v

Fixes:

  • Fix #106: match word characters to find plugins

  • Fix #101: really unref non-intermediate nodes

  • ufo-launch: query only valid graph

  • Fixed misleading documentation

  • Fixed some leaks and unreferenced resources

  • Stop num-processed from being serialized into JSON field

  • ufod: fix segfault if no address is specified

Breaks:

  • ufo-query: print errors on stderr instead of stdout

  • ufo-launch: removed –progress and –time in favor of –quiet

  • Removed public ufo_signal_emit symbol

  • Use date and time to differentiate trace profiles

  • Replace clprof with ufo-prof

Version 0.9.1

Release on January 12th 2016.

Enhancements:

  • Pass a property map to scheduler in the Python wrapper

  • Added convenience wrapper to copy into a buffer

Fixes:

  • Fix version numbering from 0.8.x to 0.9.x

  • Fix documentation issues

  • Fix build problems if libzmq is not present

  • ufo-launch unrefs non-intermediate nodes

Version 0.9

Release on November 3rd 2015.

Enhancements:

  • Generally improved debug output

  • Add support for plugin packages as demonstrated by the ART plugins

  • Add Docker installation method

  • Add simple fabfile to start ufod instances

  • Allow reductors to pause processing

  • Add ufo-query binary to retrieve info about tasks

  • Add ufo_buffer_set_device_array API call

  • Python: add ufo.numpy.empty_like

  • ufo-launch: add –dump flag to serialize to JSON

  • ufo-launch: add Bash completion script

  • ufo-launch: pass address list

Fixes:

  • Fix segfault with long-running tasks

  • Prevent daemon from leaking OpenCL resources

  • Fix broken continuous daemon operation

  • ufo-launch: fix parsing uint64

  • Use same reductor policy for both schedulers

  • Fix ufo_buffer_new_with_data

  • Fix nano second to second conversion

Breaks:

  • Specify device subset with UFO_DEVICES

  • Do not prepend . to trace and profile output

  • Change ufo_buffer_set_host_array signature

  • Use common timestamp unit for both trace types

Version 0.8

Release on May 19th 2015.

This release breaks with the distinction of ArchGraph and Resources. The former is removed with its functionality moved to the latter. Besides that numerous improvements have been incorporated:

  • ufo-launch: add –time and –trace options

  • Added a systemd unit for ufod

  • Fixed cluster communication.

  • Fixed #86: use CL_INTENSITY instead of CL_R

  • Handle kernel path with environment variable UFO_KERNEL_PATH

  • ufo-launch now parses boolean properties

  • Removed hard dependency on ZeroMQ

  • Removed WITH_DEBUG and WITH_PROFILING CMake variables

  • Add JSON-based TANGO server for remote computation

  • Removed ufo_base_scheduler_get_gpu_nodes()

  • Fixed #80: add ufo_gpu_node_get_info()

  • Fixed re-running the same task graph

  • Removed tifffile dependency

  • Fixed #78: reset num-processed before execution

  • Fixed #77: initialize threads when accessing the GIL

Version 0.7

Released on February 20th 2015.

This is a major update from the previous version released a year ago. Besides numerous bug fixes and compatibility enhancements we

  • added ufo_signal_emit for threadsafe signal emission

  • export plugin and kernel directories via pkg-config so that thirdparty plugins easily know where to install themselves

  • added the ufo-launch tool to run basic pipelines directly from the command line

  • merged the ufo-python-tools and ufo-tests repos with the core repository

  • added additional buffer depths

  • parse the UFO_PLUGIN_PATH environment variable to specify additional plugin locations

  • added a copy task which copies data from input 0

  • added a fixed scheduler for manual assignment of hardware resources to task nodes

  • added “processed” and “generated” signals to tasks which emit whenever either action completed

  • added UFO_DEVICE_TYPE_ACC for accelerator devices

  • added buffer views for larger-than-GPU data

  • output OpenCL profiler information as Chrome JSON

  • added ufo_resources_get_kernel_with_opts()

  • added ufo_buffer_set_host_array()

  • added ufo_buffer_get/set_metadata(), ufo_buffer_get_metadata_keys() and ufo_buffer_copy_meta_data()

  • added ufo_buffer_get_device_array_with_offset()

  • added ufo_buffer_get_location()

and broke compatibility by

  • retiring the UfoConfig infrastructure and

  • replacing g_message() with g_debug()

Version 0.6

Released on January 24th 2014.

Due to the inclusion of Autotools builds, we restructured and cleaned up installation paths, along the lines of GLib. That means headers are installed into major API version dependent directories, e.g. now /usr/include/ufo-0/ufo. We also split hardware-dependent and -independent files, thus kernels go into /usr/share/ufo now. In the same vein, we also lowered the SO version down to 0, so please if you have installed from source, make sure to remove any traces of an old UFO installation before installing the latest version.

Minor changes include:

  • Added “time” property to scheduler.

  • Scheduler can now use and existing arch graph and return associated resources.

  • By default only GPU devices are used.

  • Documentation is now hosted at ufo-core.readthedocs.org.

  • Added “num-processed” property.

Developers

  • Replace GList for loops with g_list_for macro

  • Include compat.h

  • Renamed and install mkfilter as ufo-mkfilter.

  • ufo-core is now monitored by Travis CI

  • Removed the python/ subdir which is replaced by ufo-python-tools

  • Removed unnecessary clprof tool.

Bugfixes

  • Fix and simplify deploy script

  • Require json-glib in pkgconfig

  • Link to documentation on rtd.org

  • Fix .gir generation

  • Fix installation path for header files

  • Do not error on deprecated declarations

Version 0.5

Released on October 28th 2013.

  • Added MPI support as an alternative to ZMQ.

  • Added basic math operations for use with filters.

  • UFO can now be used reliably in a multithreaded Python context. That means, calling ufo_scheduler_run in a Python thread will “just work”. This change allows run-time injection of NumPy buffers into the task graph.

Developers

  • Add -DDEBUG when debug is enabled so we can #ifdef it

  • Add GLib version guards

  • ufo-core compiles with Clang

  • CMake 2.6 is used solely throughout the sources

  • Add convenience function ufo_buffer_new_with_size

  • Add a shim macros to support both zmq 2 and 3

  • Add UFO_USE_GPU env var to restrict to single GPU

  • Added ufo_resource_manager_get_cached_kernel that always returns the same kernel object when given the same file and kernel name. Note, that you have to guard it properly and do not call clSetKernelArg from multiple threads at the same time.

  • Add profile tracing to produce a JSON trace event file, that can be read and visualized with Google’s Chrome browser. It can be enabled with UfoScheduler::enable-tracing set to TRUE.

Bug fixes

  • Fix #6: Don’t use enum values as bit flags

  • Fix bug: no plugin name is sent to remote nodes

  • Fix copy segfault in when source has not alloc’d

  • Removed dependency on a C++ compiler

  • Fix reduction problem

Version 0.4

Released on July 18th 2013.

Major changes

  • Rewrote internal architecture for better scheduling.

  • Remove profiler levels and add more output

  • Implement input data partitioning: On clusters where distributed data access is possible, we can achieve perfect linear scalability by partitioning the input data set.

  • Install SIGTERM handler for cleanup of node server

Features

  • Add ufo_task_graph_get_json_data

  • Streamline and simplify scheduling

  • Provide function to flatten graph

  • Provide graph copy functionality

  • Add node indices for copies

  • Add all paths as OpenCL include paths

  • Write out JSON version

  • Search in UFO_PLUGIN_PATH env var

Bug fixes

  • Fix problems with AMD platforms

  • Fix timestamp readout

  • Fix potential single integer overflow

  • Exit when JSON tasks could not be found

  • Fix remote tasks getting stuck

  • Unref expanded nodes explicitly

  • Fix #189: don’t copy nodes with more than one input

  • Fix #219: Warn instead of segfault

  • Fix annotation for older GI compiler

  • Fix problem with first remote data item

  • Fix platform selection

  • Fix problems with objects that are not unreffed

  • Refactor buffer and add support for #184

  • Refactor resources and fix #183

  • Fix buffer for broadcast operations

Version 0.3

Released on February 8th 2013.

Major breakage

  • A graph is now a simple data structure and not a specific graph of task nodes. This is implemented by a TaskGraph.

  • Filters are now called TaskNodes and connected with ufo_task_graph_connect_nodes and ufo_task_graph_connect_nodes_full respectively.

Graph expansion

With 0.2, Using multiple GPUs was possible by manually splitting paths in the graph and assigning GPUs. Now, task graphs are automatically expanded depending on the number of available GPUs and remote processing slaves that are started with the newly added ufod server software.

Minor improvements

  • A deploy.sh script has been added for easier deployment of the software stack. This is especially useful to install everything in the home directory of the user, who only needs to setup LD_LIBRARY_PATH and GI_TYPELIB_PATH correctly to run the software.

Version 0.2

Released on November 8th 2012.

Major breakage

  • Filters are now prefixed again with libfilter to allow introspected documentation. Thus, any filter built for 0.1 cannot be used because they are simply not found.

  • ufo_plugin_manager_get_filter() received a new third parameter error that reports errors when opening and loading a UfoFilter from a shared object.

  • ufo_resource_manager_add_program() is removed.

  • The kernel file name must be passed to ufo_resource_manager_get_kernel().

  • The CHECK_ERROR macro defined in ufo-resource-manager.h was renamed to CHECK_OPENCL_ERROR to better reflect its real purpose.

  • The old JSON specification has been changed to reflect the possibilities of the current API. Thus, JSON files that worked under Ufo 0.1 cannot be read with Ufo 0.2.

  • Removed the otherwise unused ufo_buffer_get_transfer_time() and replaced this with the more flexible ufo_buffer_get_transfer_timer().

  • Rename ufo_filter_initialize() to ufo_filter_set_plugin_name() that reflects its true meaning.

Scheduling

A more scheduable way to run filters has been implemented with the virtual process_cpu() and process_gpu() methods. Contrary to the old way, they act on one working set at a time that is passed as an array of pointers to UfoBuffer. Sometimes, a filter needs to setup data depending on the input size. For this reason, the virtual method initialize() takes a second parameter that is again a list of pointers to buffer objects.

Moreover, the UfoScheduler class has been added that is combining the work previously accomplished by ufo_filter_process() and ufo_graph_run(). The scheduler orchestrates the filters and assigns resources in a meaningful way.

If written in the new kernel style, producer filters must return a boolean flag denoting if data was produced or not.

General improvements

  • The manual was restructured considerably.

  • Saving graphs as JSON files has been added via ufo_graph_save_to_json().

  • Filters can now wait until their properties satisfy a condition using ufo_filter_wait_until(), see also Wait until a property satisfies a condition.

  • A new method ufo_resource_manager_get_kernel_from_source() so that filters can load kernels directly from source.

  • Streamlined error handling: Filters should not issue g_warnings or g_errors on their own anymore but create an error with g_error_new and return that.

Version 0.1.1

  • Ticket #55: tests/test-channel blocks indefinitely