% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/maestro_tags.R
\name{maestro_tags}
\alias{maestro_tags}
\title{Maestro Tags}
\description{
maestro tags are roxygen2 comments for configuring the scheduling and execution
of pipelines.
}
\details{
maestro tags follow the format: \verb{#' @maestroTagName value}

Some tags may not take a value.

maestro tags must be written above the function that is to be included as a pipeline.
A typical pipeline with tags could look like this:

\if{html}{\out{<div class="sourceCode">}}\preformatted{#' @maestroFrequency 1 hour
#' @maestroStartTime 12:30:00
#' @maestroLogLevel WARN
my_pipeline <- function() \{

   # Pipeline code
\}
}\if{html}{\out{</div>}}

Below are descriptions of all the tags currently available in maestro along
with examples.
}
\section{maestroFrequency}{
How often to run the pipeline. This tag takes a time unit indicating how long
to wait between subsequent runs of the pipeline. Acceptable values include
an integer value followed by one of minute(s), hour(s), day(s), week(s), month(s),
and year(s). Note that some combinations of integer + unit may be invalid.
Adverbs like 'hourly', 'daily', 'weekly', etc. are also valid.

Default: \verb{1 day}

Examples:
\itemize{
\item \verb{#' @maestroFrequency 1 hour}
\item \verb{#' @maestroFrequency daily}
\item \verb{#' @maestroFrequency 2 weeks}
\item \verb{#' @maestroFrequency 3 months}
}
}

\section{maestroStartTime}{
Timestamp, date, or time corresponding to the start time of the pipeline. This
also sets the cadence of the pipeline in some cases. For instance, if the start time
is \verb{2025-03-18 03:00:00} and the frequency is daily, the pipeline will run at 03:00
every day. A value in the future prevents the pipeline from running until that time
has been reached.

For pipelines with a frequency lower than daily, partial anchor formats are supported
to make it easier to express natural cycle points without choosing a specific date:
\itemize{
\item \strong{\code{HH:MM:SS}} — time-of-day anchor for \code{minute}, \code{hour}, or \code{day} frequencies.
e.g., \code{08:00:00} runs every day at 8am.
\item \strong{\verb{Mon HH:MM:SS}} (weekday abbreviation + time) — anchor for \code{week} or multi-week
frequencies. Resolved to that weekday of the current ISO week.
e.g., \verb{Mon 04:00:00} with \verb{@maestroFrequency 1 week} runs every Monday at 4am.
Valid abbreviations: \code{Mon}, \code{Tue}, \code{Wed}, \code{Thu}, \code{Fri}, \code{Sat}, \code{Sun}.
\item \strong{\verb{DD HH:MM:SS}} or \strong{\code{DD}} (month-day + optional time) — anchor for \code{month}
frequencies. Resolved to that day of the current month.
e.g., \verb{15 04:00:00} with \verb{@maestroFrequency 1 month} runs on the 15th of every month at 4am.
}

Default: \verb{2024-01-01 00:00:00}

Examples:
\itemize{
\item \verb{#' @maestroStartTime 2025-02-05 12:00:00}
\item \verb{#' @maestroStartTime 2025-01-01}
\item \verb{#' @maestroStartTime 08:00:00}
\item \verb{#' @maestroStartTime Mon 04:00:00}
\item \verb{#' @maestroStartTime Wed 09:30:00}
\item \verb{#' @maestroStartTime 15 04:00:00}
\item \verb{#' @maestroStartTime 1}
}
}

\section{maestroTz}{
Timezone in which the maestroStartTime is to be considered. Takes any valid timezone
string that can be found in \code{OlsonNames()}.

Default: \code{UTC}

Examples:
\itemize{
\item \verb{#' @maestroTz Europe/Paris}
\item \verb{#' @maestroTz America/Halifax}
\item \verb{#' @maestroTz US/Pacific}
}
}

\section{maestroLogLevel}{
Minimum logging threshold for messages, warnings, and errors that come from the pipeline.
These levels correspond to those in \code{logger:::log_levels_supported} but most commonly
used are ERROR, WARN, INFO. For example, if you use \code{WARN} then any messages of lower
urgency (i.e., INFO) will be suppressed, but errors will be logged.

Default: \code{INFO}

Examples:
\itemize{
\item \verb{#' @maestroLogLevel ERROR}
\item \verb{#' @maestroLogLevel WARN}
}
}

\section{maestroHours}{
Specific hours of the day in which to run the pipeline. This only applies for pipelines that run
at an hourly or minutely frequency. Acceptable values are integers from 0-23 separated
by spaces. If empty, pipeline runs at all hours. This tag uses the timezone specified by
\code{maestroTz} (will be UTC if empty).

Default:

Examples:
\itemize{
\item \verb{#' @maestroHours 1 4 7 10}
\item \verb{#' @maestroHours 0 5 20}
}
}

\section{maestroDays}{
Specific days of week or days of month on which to run the pipeline. This only applies
for pipelines that run at a minutely, hourly, or daily frequency. Acceptable values are
either integers from 1-31 or day of week strings like Mon, Tue, Wed, etc. These two
options cannot be used in combination.

Default:

Examples
\itemize{
\item \verb{#' @maestroDays 1 11 21 31}
\item \verb{#' @maestroDays Mon Tue Wed Thu Fri}
}
}

\section{maestroMonths}{
Specific months of the year on which to run the pipeline. This only applies for
pipelines that do run at least monthly. Acceptable values are integers (1-12) corresponding
to the month of the year (e.g., 1 = January, 2 = February, etc.).

Default:

Examples:
\itemize{
\item \verb{#' @maestroMonths 3 8 12}
\item \verb{#' @maestroMonths 10}
}
}

\section{maestroInputs}{
For a DAG style pipeline, the names of pipelines that input into the pipeline.
These names must match the function name defining the inputting pipeline. Multiple
pipelines can be used as inputs and the input value is used in the target pipeline
via the required \code{.input} parameter. Note that this tag could be redundant if the
inputting pipeline uses \code{maestroOutputs}.

To enable \strong{fan-in (collect)}, wrap one or more input names with \code{collect()}.
The downstream pipeline receives a named list as \code{.input}, where each name
corresponds to an upstream pipeline and each value is that pipeline's return
value. All listed upstream pipelines must have succeeded before the collect
pipeline fires. Works with both static multi-source inputs and dynamic fan-out
followed by fan-in (\verb{@maestroMap} → \code{collect()}): in the latter case \code{.input}
is a list of all successful iteration results.

Default:

Examples:
\itemize{
\item \verb{#' @maestroInputs extract verify}
\item \verb{#' @maestroInputs collect(letter_a, letter_b)}
\item \verb{#' @maestroInputs collect(multiply)} \emph{(collect all iterations of a \verb{@maestroMap} upstream)}
}
}

\section{maestroOutputs}{
For a DAG style pipeline, the names of pipelines that receive the return value of
this pipeline as input. These names must match the function name defining the
outputting pipeline. Multiple pipelines can be outputted into. The return value
of the pipeline will be passed into the receiving pipeline. Note that this tag
could be redundant if pipeline to be inputted into uses \code{maestroInputs}.

Default:

Examples:
\itemize{
\item \verb{#' @maestroOutputs transform}
}
}

\section{maestroSkip}{
Flags a pipeline to never be executed even if it is scheduled to run. This can
be useful when developing or testing a pipeline. This tag takes no value, instead
the presence of the tag indicates whether it is skipped. This tag is ignored when
\code{run_schedule(..., run_all = TRUE)} or when using \code{invoke()}.

Default:

Examples:
\itemize{
\item \verb{#' @maestroSkip}
}
}

\section{maestroRunIf}{
An R expression that is evaluated at run time to determine whether the pipeline
should execute. The expression must return a single \code{TRUE} or \code{FALSE}. If the
expression returns \code{FALSE}, the pipeline is silently skipped for that run. Resources
passed via \code{run_schedule(..., resources = list(...))} are available inside the
expression, as is \code{.input} for DAG pipelines.

Default:

Examples:
\itemize{
\item \verb{#' @maestroRunIf Sys.getenv("RUN_PIPELINE") == "true"}
\item \verb{#' @maestroRunIf lubridate::wday(lubridate::now()) == 2}
\item \verb{#' @maestroRunIf !is.null(.input)}
}
}

\section{maestroMap}{
Enables \strong{dynamic fan-out (scatter)}: the downstream pipeline executes once
per element of the upstream return value. When the tag value is empty, each
element of the upstream return value is passed directly as \code{.input}. When one
or more R expressions referencing \code{.input} fields are provided (space-separated),
those fields are scattered over and zipped together \code{pmap}-style: each iteration
receives \code{.input} with all specified fields replaced by their i-th element.
The full list remains accessible as \code{.input} in every branch.

Vectors must all have the same length, unless a vector has length 1, in which
case it is recycled across all iterations. Mismatched lengths produce a
pipeline error.

Requires \verb{@maestroInputs} to reference an upstream pipeline.

Default:

Examples:
\itemize{
\item \verb{#' @maestroMap} \emph{(iterate over each element of the upstream return value)}
\item \verb{#' @maestroMap .input$items}
\item \verb{#' @maestroMap .input$ids .input$labels}
\item \verb{#' @maestroMap .input$model .input$label .input$threshold}
}
}

\section{maestroPriority}{
Determines the order in which pipelines that run at the same scheduled instance
are executed. Values are positive integers from 1 to N. Order is determined in
descending order such that 1 indicates the highest priority. Pipelines with the
same priority run in the order in which \code{build_schedule()} parses the pipeline
(usually alphabetical according to file name and then line number within file).
By default, all pipelines are given equal priority.

Default:

Examples:
\itemize{
\item \verb{#' @maestroPriority 1}
\item \verb{#' @maestroPriority 3}
}
}

\section{maestroFlags}{
Arbitrary strings which are then made accessible via \code{get_flags()}. A pipeline
can have multiple tags separated by spaces.

Default:

Examples:
\itemize{
\item \verb{#' @maestroFlags critical etl cloud}
\item \verb{#' @maestroFlags aviation}
}
}

\section{maestroLabel}{
Key-value pairs used for attributing metadata for a pipeline. Multiple labels
can be assigned to a pipeline using multiple instances of \verb{@maestroLabel}. These
labels can be extracted as a data.frame. Each label must follow the pattern of
\code{key} \code{value}.

Default:

Examples:
\itemize{
\item \verb{#' @maestroLabel domain transportation}
\item \verb{#' @maestroLabel author will.hipson}
}
}

\section{maestro}{
Generic tag for identifying a maestro pipeline with all the defaults. Useful
when you want a pipeline to be scheduled via maestro that accepts all default
tag values. Only use this tag if you have no other maestro tags. The tag takes
no value.

Default:

Examples:
\itemize{
\item \verb{#' @maestro}
}
}

