Changelog¶
All notable changes to this project will be documented in this file.
Updates should follow the Keep a CHANGELOG principles.
This project adheres to Semantic Versioning. For major version 0, we will increment the minor version for backward-incompatible changes.
0.7.0 - 2019-06-25¶
Changed¶
- The former
Transformer
interface is now namedPropertyTransformer
, to distinguish it from the newRecordTransformer
interface. - Property transformer classes have been moved to the
Soong\Transformer\Property
namespace. - The structure of sample task configurations has been changed to use an array of
RecordTransformer
instances under thetransform
key. - Transformer construction has been moved from the
EtlTask
implementation to theEtlCommand
implementation - thetransform
option forEtlTask
now requires an array ofRecordTransformer
instances. PropertyTransformer::transform()
no longer specifies the types of its argument and return value. Implementations should specify their specific return type, and validate the expected argument type.Record::getProperty()
renamed togetPropertyValue()
,setProperty()
renamed tosetPropertyValue()
.- Symfony Console dependency has been loosened to allow versions 3.4 through 4.x.
- Release checklist added to CONTRIBUTING
Removed¶
- The
Property
andPropertyFactory
interfaces and implementations have been removed - all code using properties now uses values directly, leaving type-checking to PHP.property_factory
configuration options have accordingly been removed.
Added¶
- The
RecordTransformer
interface has been added. - The
PropertyMapper
record transformer class has been added. - The
Copy
record transformer class has been added. - The
--limit
option has been added to themigrate
androllback
commands. - Transformer exceptions added.
0.6.0 - 2019-05-01¶
Changed¶
EtlTask
now accepts its extract, key_map, and load components as object instances rather than constructing them from configuration.DataProperty
interface renamed toProperty
, andDataRecord
interface renamed toRecord
.- Derivatives of
ExtractorBase
now must accept arecord_factory
configuration option, which is an instance ofRecordFactory
. EtlTask
replaced the stringrecord_class
withRecordFactory
instancerecord_factory
.
Added¶
- The
Filter
interface has been added, to determine whether a DataRecord should be processed. - The
Select
filter has been added, allowing for filtering by comparing DataRecord properties to values using PHP comparison operators. - The
--select
option has been added to themigrate
command, allow for ad-hoc filtering of extracted data at runtime. PropertyFactory
andRecordFactory
interfaces/classes added for creation ofProperty
andRecord
instances.- Added basic console command tests.
property_factory
configuration option added toEtlTask
,LoaderBase
.ExtractorException
,KeyMapException
, andLoaderException
classes added.- Unit test for
Record
added.
0.5.3 - 2019-04-12¶
Changed¶
- Things are now configured to generate the API documentation using Doxygen on readthedocs - the generated docs are no longer kept in the repo.
0.5.2 - 2019-04-05¶
Changed¶
addTask
now takes an existing Task object instead of a class and configuration.- Static create() methods removed from all components and constructors made public.
- Static methods removed from
Task
component and moved to non-static methods on the newTaskPipeline
component:addTask()
,getTask()
,getAllTasks()
.
Added¶
ConfigurableComponent
interface added, and all configurable component interfaces inherit from it.OptionsResolverComponent
added implementingConfigurableComponent
usingSymfony\OptionsResolver
- this is now the base class for all configurable components. Any such components adding configuration options to their parent class must implementoptionDefinitions()
to defined them.- Commands now use
hassankhan/config
instead of custom YAML handling - configuration now can be YAML, JSON, or XML transparently (examples provided for each). TaskPipeline
component for managing groups of tasks.ComponentNotFound
andDuplicateTask
exceptions added.- Tests for
Extractor
,KeyMap
,Loader
, andTask
components. - Tests for
KeyMapLookup
component. - Smoke test to make sure all provided examples keep working.
Removed¶
isCompleted
method onTask
- unneeded until we add dependencies.
0.4.0 - 2019-02-15¶
Added¶
EtlTaskInterface::getLoader()
- Tests for
Data
andTransformer
components.
Changed¶
- DBAL and Csv implementations have been moved:
Soong\Csv\Extractor
->Soong\Extractor\Csv
Soong\Csv\Loader
->Soong\Loader\Csv
Soong\DBAL\Extractor
->Soong\Extractor\DBAL
Soong\DBAL\KeyMap
->Soong\KeyMap\DBAL
Soong\DBAL\Loader
->Soong\Extractor\Loader
Interface
andTrait
suffixes removed from all interfaces and traits.- All interfaces moved into
Contracts
directory. - All main components must now be created using
Class::create()
rather thannew
. This affects:DataPropertyInterface
DataRecordInterface
ExtractorInterface
KeyMapInterface
LoaderInterface
TaskInterface
TransformerInterface
- Explicit component documentation pages have been removed in favor of Doxygen-generated documentation.
- Existing inline documentation has been cleaned up and expanded.
Removed¶
KeyMapFactory
MemoryKeyMap
0.3.0 - 2019-02-05¶
Added¶
- Added
getExtractor()
andgetAllTasks()
to task interfaces/implementations. - Initial implementation of the
status
console command. - All documentation moved into this repo, will be available on readthedocs.io.
DataPropertyInterface::isEmpty()
DataRecordInterface::propertyExists()
Removed¶
- Removed subtask concept from task interfaces/implementations.
- Removed
CountableExtractorInterface
.
Changed¶
DataRecordInterface::setProperty()
no longer accepts null values (only property objects with null values).DataRecordInterface::getProperty()
no longer returns null values (only property objects with null values).TransformerInterface::transform()
no longer accepts or returns null values (only property objects with null values).- The
$configuration
parameter toTransformerInterface::transform()
has been removed - configuration should be passed in the constructor instead. - SHA256 for key hashing in
KeyMapBase
. - Added configuration key for hash algorithm.
Fixed¶
- Hashing serialized keys needs to make sure values are always strings.