How to do daily load in Hive

[DataFlair Team]

every time if you have to load the data every day so do we have to run query manually every day?

[DataFlair Team]

One way to load the data every day in Hive is to create new partitions for incremental loads with the date as a partition key. It will avoid the creation of new tables for each incremental load. Moreover, querying and reloading of data will be easier, with a single table partitioned appropriately. Furthermore, unless required, try to avoid nested sub-partitions. As an example, create YYYY-MM-DD as the single partition, rather than creating year, month, and day as nested partitions.

[Author]

Posts