Release Notes - Apache Griffin 0.6.0 ** Highlights ** * Enhancing Griffin's functionality and broadening application scenarios. * Fixing some bugs and improving code quality. * Adding new features to meet requirements from the Griffin community. ** New Feature * [GRIFFIN-256] - Alternatives way to access hive metadata * [GRIFFIN-289] - new feature for griffin COMPLETENESS dq type * [GRIFFIN-311] - keep null value columns optionally * [GRIFFIN-339] - Import griffin tool for debug and run user jobs ** Improvement * [GRIFFIN-240] - [Service] Return JobInstance in response to a call Trigger job by id * [GRIFFIN-247] - Support pluggable sinks * [GRIFFIN-253] - How to implement the HA in apache griffin? * [GRIFFIN-263] - Use independent logger for griffin * [GRIFFIN-264] - Clean up maven dependencies in pom.xml * [GRIFFIN-267] - Loggable trait use Scala call by name for lazy evaluation * [GRIFFIN-269] - Griffin submit job to livy with Kerberos Authentication * [GRIFFIN-273] - Griffin Service with user-defined log4j2 configuration * [GRIFFIN-278] - AvroBatchDataConnector handle input is directory * [GRIFFIN-279] - Upgrade Spring boot to 2.1.7.RELEASE * [GRIFFIN-295] - Limit the memory used by test case * [GRIFFIN-298] - Add CompletenessExpr2DQSteps transform test suite * [GRIFFIN-299] - Add oracle jdk8 support in travis build phase * [GRIFFIN-301] - Update custom data connector to have the same parameters as build-in data connector * [GRIFFIN-304] - Remove all traces of different Contexts in favour of SparkSession * [GRIFFIN-310] - Format scala code according to scalastyle-config.xml and enable scala code style checking by default * [GRIFFIN-312] - Code Style Standardization * [GRIFFIN-317] - Define guidelines for Griffin Project Improvement Proposals (GPIP) * [GRIFFIN-319] - Deprecate old Data Connectors * [GRIFFIN-322] - Add SQL mode for ES connector * [GRIFFIN-323] - Refactor configuration for Data Source and Data Source Connector ** Bug * [GRIFFIN-266] - [Service] Measure's rules are not always properly sorted * [GRIFFIN-270] - Fix wrong hadoop configuration in deploy guide * [GRIFFIN-274] - Error: java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class * [GRIFFIN-275] - [Measure] Post data to ElasticSearch failed * [GRIFFIN-293] - [Service] livy.need.queue=true * [GRIFFIN-294] - generate wrong completeness sql * [GRIFFIN-296] - Update .travis.yml to specify Maven Memory * [GRIFFIN-300] - MySqlDataConnector isn't work * [GRIFFIN-306] - FileBasedDataConnector doesn't translate the date in the paths * [GRIFFIN-316] - Spark runtime exception cannot be caught while running a dq application * [GRIFFIN-318] - Replace all YYYY with yyyy in all user guides and examples * [GRIFFIN-321] - visit service web page return 404 * [GRIFFIN-337] - evaluation rule for uniqueness * [GRIFFIN-329] - Measure unit test cases failed on the condition of no docker image * [GRIFFIN-348] - Change connector schema in data source ** Task * [GRIFFIN-271] - Before 0.6.0 milestone release, We need to check and trim current codes against apache griffin dev guide * [GRIFFIN-272] - Abort the application if data source fail to load data. * [GRIFFIN-276] - Add batch type measure job test case * [GRIFFIN-281] - Add custom sink testcase * [GRIFFIN-283] - Move sink steps into TransformStep * [GRIFFIN-286] - Remove spark-testing-base dependency jar * [GRIFFIN-287] - Use Jackson to parse json data * [GRIFFIN-288] - Optimize hdfs sink * [GRIFFIN-290] - Fix bug for submitting job to livy * [GRIFFIN-291] - Relocate HttpClient code in measure jar using shade plugin * [GRIFFIN-297] - GRIFFIN-302 Allow support for additional file based data sources * [GRIFFIN-305] - GRIFFIN-303 Standardize Sink Hierarchy * [GRIFFIN-315] - GRIFFIN-302 Allow support for JDBC based data connector * [GRIFFIN-326] - GRIFFIN-302 New implementation for Elasticsearch Data Connector (Batch)