kartothek.io_components.cube.remove module

kartothek.io_components.cube.remove.prepare_metapartitions_for_removal_action(cube, store, conditions, ktk_cube_dataset_ids, existing_datasets)

Prepare MetaPartition to express removal of given data range from cube.

The MetaPartition must still be written using mp.store_dataframes(...) and added to the Dataset using a kartothek update method.

Parameters

• cube (kartothek.core.cube.cube.Cube) – Cube spec.

• store (Union[simplekv.KeyValueStore, Callable[[], simplekv.KeyValueStore]]) – Store.

• conditions (Union[None, Condition, Iterable[Condition], Conjunction]) – Conditions that should be applied, optional. Defaults to “entire cube”.

• ktk_cube_dataset_ids (Optional[Union[Iterable[str], str]]) – Ktk_cube dataset IDs to apply the remove action to, optional. Default to “all”.

• existing_datasets (Dict[str, kartothek.core.dataset.DatasetMetadata]) – Existing datasets.

Returns

metapartitions – kartothek.io_components.metapartition.MetaPartition, List[Dict[str, Any]]]] MetaPartitions that should be written and updatet to the kartothek datasets as well as the delete_scope for kartothek.

Return type

Dict[str, Tuple[kartothek.core.dataset.DatasetMetadata,