dbsp: checkpointer: write empty checkpoint catalog on prepare

When preparing to write a checkpoint, make an empty checkpoint catalog.

The `Checkpointer::read_checkpoints` method raises an error if there is
a checkpoint UUID dir without a checkpoint catalog. This means when
creating the first checkpoint, the operators start writing to the
checkpoint dir, and the checkpoint catalog hasn't been created yet. When
apps call `/checkpoints` or any code path that calls `read_checkpoints`
is triggered, it will raise an error saying it found an unexpected
checkpoint dir without a checkpoint catalog.

Signed-off-by: Abhinav Gyawali <22275402+abhizer@users.noreply.github.com>

[web-console] Format using new 'force brackets in conditions' rule

Signed-off-by: Karakatiza666 <bulakh.96@gmail.com>

[ci] apply automatic fixes

Signed-off-by: feldera-bot <feldera-bot@feldera.com>

[dbsp] Merge the inputs in shard_pairs().

shard_pairs() takes a collection of sorted input vectors and shards them.
The output vectors must also be sorted, which means that the code must
merge them to avoid jumbling data in the output vectors.  However, it
didn't do that.  This commit fixes the problem.

Only multihost pipelines use this code.

Signed-off-by: Ben Pfaff <blp@feldera.com>

[web-console] Fix page reload loop when feldera version upgraded in b…

…ackground

Signed-off-by: Karakatiza666 <bulakh.96@gmail.com>

[web-console] Implement Pipeline Health Monitoring UI

Improve formatting of date ranges in shared Health UI

Make inline drawers become overlays on smaller screens

Refactor shared Health UI componetns to be more generic

Signed-off-by: Karakatiza666 <bulakh.96@gmail.com>

pipeline-manager: pipeline monitoring events

Both at a regular interval and whenever the status of the pipeline
changes, store this as an event in the database. If there are no
changes, every 10 minutes it stores a duplicate event. If there are only
changes to the details, it only updates it every 10 seconds. However, if
there are changes to the status, it can update more frequently than
every 10 seconds.

A cleanup threads runs in the background, waking up every 30 seconds to
clean up for each pipeline events exceeding the retention limit of (by
default) 720. The retained events as such span approximately between 2
hours and 5 days generally. The number of retained events per pipeline
can be changed via CLI argument `--pipeline-monitor-events-retention`
or its corresponding environment variable
`FELDERA_PIPELINE_MONITOR_EVENTS_RETENTION`.

API endpoints:
- `GET /v0/pipelines/<pipeline>/events?selector=all/status`
- `GET /v0/pipelines/<pipeline>/event/latest?selector=all/status`
- `GET /v0/pipelines/<pipeline>/event/<event-id>?selector=all/status`

The clients have been updated with corresponding functionality:

- CLI:
  - `fda events <pipeline> <selector>`
  - `fda event <pipeline> latest <selector>`
  - `fda event <pipeline> <event-id> <selector>`

- Python:
  - `pipeline.events(selector=...)`
  - `pipeline.event("latest", selector=...)`
  - `pipeline.event(event_id, selector=...)`

Signed-off-by: Simon Kassing <simon.kassing@feldera.com>

[SQL] Increase the macro expansion recursion size for operators with …

…wide star joins

Signed-off-by: Mihai Budiu <mbudiu@feldera.com>

ci: Automate dbt-feldera release

Signed-off-by: Gerd Zellweger <mail@gerdzellweger.com>

[adapters] Fix comment in controller code.

7488e67 ("[adapters] Divide Feldera checkpoints into multiple
phases.") eliminated waiting for output to complete as a reason for
preventing the pipeline from running.

Signed-off-by: Ben Pfaff <blp@feldera.com>