LTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe Diffusion-based Planning

Abstract

Operating effectively in complex environments while complying with specified constraints is crucial for the safe and successful deployment of robots that interact with and operate around people. In this work, we focus on generating long-horizon trajectories that adhere to static and temporally-extended constraints/instructions at test time. We propose a data-driven diffusion-based framework, LTLDoG, that modifies the inference steps of the reverse process given an instruction specified using finite linear temporal logic ($\textnormal{LTL}_{f}$). LTLDoG leverages a satisfaction value function on $\textnormal{LTL}_{f}$ and guides the sampling steps using its gradient field. This value function can also be trained to generalize to new instructions not observed during training, enabling flexible test-time adaptability. Experiments in robot navigation and manipulation illustrate that the method is able to generate trajectories that satisfy formulae that specify obstacle avoidance and visitation sequences. Code and supplementary material are available online at https://github.com/clear-nus/ltldog.

I Introduction

Recent methodologies [1, 2, 3] utilizing data-driven diffusion models [4, 5, 6] have shown remarkable performance in generating robot behaviors across a wide range of tasks. Thanks to their ability to model complex distributions, these methods have surpassed several leading offline reinforcement learning techniques and classical model-based trajectory optimization methods, especially in long-horizon decision-making tasks [1, 3]. However, while conventional diffusion models excel at learning from training datasets, they lack the ability to adapt to new objectives or comply with new constraints during deployment. This shortcoming can lead to unsafe behaviors, posing risks to humans, robots, and their surrounding environment.

In view of this limitation, there has been very recent work on diffusing safe trajectories. Xiao et al. [7] integrated a dynamics model into the denoising diffusion process and incorporated a class of Control Barrier Functions (CBF) to meet safety criteria. Botteghi et al. [8] approached the issue by embedding both safety and reward considerations into a constrained optimization framework, employing CBF constraints as labels for classifier guidance. However, these approaches primarily address static environmental constraints. For example, while they can maneuver around obstacles on a local scale, they fail to comply with more complex temporally-extended directives such as “avoid the kitchen until you are clean”.

Figure 1: We present LTLDoG, a diffusion-based planning framework for generating trajectories that comply with specified $\textnormal{LTL}_{f}$ formulae. In the example above, a robot dog is tasked to arrive at the goal position (A), but first has to visit B and avoid obstacles (crosses).

In this work, we propose an alternative approach to flexible trajectory planning with diffusion models, designed to satisfy both static safety requirements and temporal constraints. The core idea is to plan with diffusion models to satisfy finite linear temporal logic ($\textnormal{LTL}_{f}$) formulae [9]. $\textnormal{LTL}_{f}$ offers the ability to define a broad spectrum of instructions/constraints that might emerge during deployment. For example, $\textnormal{LTL}_{f}$ can describe a visitation order of different objects and locations. The use of propositional logic operators, such as not, facilitates the delineation of safe regions within the state space.

We develop LTLDoG (LTL Diffusion-orienting Guidance, pronounced “Little Dog”), a posterior-sampling based diffusion framework that accommodates finite LTL formulae at test time. We present two variants of LTLDoG: our main method, LTLDoG-S, can be applied to the generation of finite-length trajectory in robot tasks where the labeling function for propositional events is differentiable. LTLDoG-S employs a differentiable formula checker in conditional sampling — specifically, we modify the reverse process to condition upon the criteria that the final (predicted) trajectory satisfies a given $\textnormal{LTL}_{f}$ formula. For when a differentiable labeling function is unavailable, we propose LTLDoG-R, which uses a trained $\textnormal{LTL}_{f}$ neural-symbolic evaluator for posterior sampling. Notably, both variants do not require collecting expert demonstrations for every potential $\textnormal{LTL}_{f}$ instruction. They retain the temporal compositionality and local consistency properties associated with diffusion models [1] — as long as the dataset contains a diverse set of paths, they can potentially “stitch together” snippets of trajectories from the training data to generate plans for unseen $\textnormal{LTL}_{f}$ formulae.

Experiments on two benchmark environments (long-horizon planning for navigation and policy learning for manipulation) demonstrate that LTLDoG is able to generate trajectories that satisfy feasible safety and temporal constraints. We find that our methods possess the ability to re-plan alternative paths at a high-level based on given instruction. Moreover, real robot experiments show that the generated trajectories can be successfully transferred to a quadruped robot. In summary, this paper makes three key contributions:

• A conditional trajectory sampling approach designed for $\textnormal{LTL}_{f}$ instructions that leverages pre-trained diffusion models;

• A regressor-guidance neural network for diffusion that generalizes to novel $\textnormal{LTL}_{f}$ formulae within a given template structure;

• Experimental results on benchmark problems and real world demonstrations that validate the effectiveness of planning with safety and temporal constraints.

From a broader perspective, LTLDoG is the first method that fuses symbolic model checking (using $\textnormal{LTL}_{f}$) with expressive diffusion-based generative models. We hope our results lays the groundwork towards performant, yet safer and more trustworthy robots.

II Preliminaries and Notation

In this work, our focus is to extend diffusion-based planning methods towards generating trajectories that comply with specified $\textnormal{LTL}_{f}$ formulae. Here, we provide a concise introduction to diffusion methods in the context of planning and finite linear temporal logic.

II-A Planning with Diffusion

Many tasks in planning, reinforcement learning, and imitation learning require generating trajectories under some specific objective. Let $\mathcal{S}$ and $\mathcal{A}$ denote the state and action space, respectively. We use $\bm{\tau}=\left(\mathbf{s}_{0},\mathbf{a}_{0},\mathbf{s}_{1},\mathbf{a}_{1},\ldots,\mathbf{s}_{T},\mathbf{a}_{T}\right)$ to refer to a trajectory, where $T$ is the planning horizon. The environment transitions to a new state $\mathbf{s}_{t+1}$ when an agent executes action $\mathbf{a}_{t}$ at state $\mathbf{s}_{t}$. Let the abbreviation $\mathcal{J}(\bm{\tau}|g)$ denote the objective value function conditioned on a goal state where the trajectory must terminate at, for example, (discounted) cumulative rewards in reinforcement learning, cumulative error of actions in imitation learning, or cost for safety constraints.

Diffusion-based planning methods directly generate partial or entire trajectories by using diffusion models pre-trained on a dataset of trajectories. Let $p_{0}\left(\bm{\tau}^{0}\right)$ denote the distribution of trajectories in dataset, where $\bm{\tau}^{0}$ represents a noiseless trajectory. Given an $N$-step discrete approximation of forward diffusion process $p\left(\bm{\tau}^{i}|\bm{\tau}^{i-1}\right)$ that slowly corrupts data by adding prespecified noise, diffusion models learn an iterative denoising procedure by approximating the score function $\nabla_{\bm{\tau}^{i}}\log p_{i}\left(\bm{\tau}^{i}\right)$ using a step-dependent neural network $s_{\theta}$ trained with denoising score matching [10]:

$$\theta^{*}=\operatorname*{arg\,min}_{\theta}\mathbb{E}_{i,\bm{\tau}^{i},\bm{\tau}^{0}}\left[\left\lVert s_{\theta}\left(\bm{\tau}^{i},i\right)-\nabla_{\bm{\tau}^{i}}\log p\left(\bm{\tau}^{i}|\bm{\tau}^{0}\right)\right\rVert^{2}\right],$$ (1)

in which $i\sim\mathcal{U}\{1,2,\ldots,N\}$ is the diffusion timestep, and $\bm{\tau}^{i}\sim p\left(\bm{\tau}^{i}|\bm{\tau}^{0}\right)$ is the trajectory $\bm{\tau}^{0}$ corrupted with noise. Throughout the paper, we adopt Denoising Diffusion Probabilistic Models (DDPM) [5] as the sampling method, where $p\left(\bm{\tau}^{i}|\bm{\tau}^{0}\right)=\mathcal{N}\left(\sqrt{\bar{\alpha}_{i}}\bm{\tau}^{0},\left(1-\bar{\alpha}_{i}\right)\bm{I}\right)$, $\bar{\alpha}_{i}\mathrel{\mathop{:}}=\prod_{j=1}^{i}\alpha_{i}$, $\alpha_{i}\mathrel{\mathop{:}}=1-\beta_{i}$ and $\{\beta_{i}\}$ is a sequence of positive noise scales $0<\beta_{1},\beta_{2},\dots,\beta_{N}<1$.

II-B Linear Temporal Logic (LTL)

Given a finite set of propositional symbols $\mathcal{P}$, the formula set $\Psi$ of $\textnormal{LTL}_{f}$ contains formulas recursively defined in Backus-Naur form as follows [11, 12]:

$$\varphi\mathrel{\mathop{:}}=p\;|\;\neg\varphi\;|\;\varphi\wedge\psi\;|\;\bigcirc\varphi\;|\;\varphi\operatorname{\mathsf{U}}{}\psi,$$

where $p\in\mathcal{P}$ and $\varphi,\psi\in\Psi$. Intuitively, the formula $\bigcirc\varphi$ (next $\varphi$) is satisfied if $\varphi$ is satisfied at the next time step. $\varphi\operatorname{\mathsf{U}}{}\psi$ ($\varphi$ until $\psi$) is satisfied if $\varphi$ is satisfied until $\psi$ is satisfied, and $\psi$ is satisfied by the end of the sequence. From these, other commonly used logical connectives and temporal operators can be defined according to the following equivalences: $\varphi\vee\psi=\neg\left(\neg\varphi\wedge\neg\psi\right)$, $\Diamond\varphi=\mathsf{true}{}\operatorname{\mathsf{U}}{}\varphi$ (eventually $\varphi$) and $\Box\varphi=\neg\Diamond\left(\neg\varphi\right)$ (always $\varphi$). The symbols $\mathsf{true}{}$ and $\mathsf{false}{}$ can also be in the formula set defined by $\mathsf{true}{}=\varphi\vee\neg\varphi$ and $\mathsf{false}{}=\neg\mathsf{true}{}$.

In contrast to propositional logic, these formulas are evaluated over finite sequences of observations $\bm{\sigma}=\langle\sigma_{0},\sigma_{1},\sigma_{2},\ldots\sigma_{T}\rangle$ (i.e., truth assignments to the propositional symbols in $\mathcal{P}$), where $\sigma_{t}\in\{0,1\}^{|\mathcal{P}\rvert}$ and $\sigma_{t,p}=1$ iff proposition $p\in\mathcal{P}$ is satisfied at time step $t$. $\mathsf{true}{}$ ($\mathsf{false}{}$) is always satisfied (not satisfied) by any assignment. Formally, $\bm{\sigma}$ satisfies $\varphi$ at time $t\geq 0$, denoted by $\langle{\bm{\sigma},t}\rangle\models\varphi$, as follows:

• $\langle{\bm{\sigma},t}\rangle\models p$ iff $\sigma_{t,p}=1$, where $p\in\mathcal{P}$

• $\langle{\bm{\sigma},t}\rangle\models\neg\varphi$ iff $\langle{\bm{\sigma},t}\rangle\not\models\varphi$

• $\langle{\bm{\sigma},t}\rangle\models(\varphi\wedge\psi)$ iff $\langle{\bm{\sigma},t}\rangle\models\varphi$ and $\langle{\bm{\sigma},t}\rangle\models\psi$

• $\langle{\bm{\sigma},t}\rangle\models\bigcirc\varphi$ iff $\langle{\bm{\sigma},t+1}\rangle\models\varphi$

• $\langle{\bm{\sigma},t}\rangle\models\varphi\operatorname{\mathsf{U}}{}\psi$ iff $\exists t_{2}\in[t,T]$ s.t. $\langle{\bm{\sigma},t_{2}}\rangle\models\psi$ and $\forall t_{1}\in[t,t_{2})$, $\langle{\bm{\sigma},t_{1}}\rangle\models\varphi$

A sequence $\bm{\sigma}$ is then said to satisfy $\varphi$, i.e., $\bm{\sigma}\models\varphi$, iff $\langle{\bm{\sigma},0}\rangle\models\varphi$.

III Method

In this section, we describe our primary contribution, LTLDoG, a diffusion-based framework for generating trajectories that satisfy $\textnormal{LTL}_{f}$ formulae. We first discuss how to conditionally sample using diffusion models, followed by how $\textnormal{LTL}_{f}$ formulae can be used to guide the diffusion process.

III-A Conditional Sampling in Diffusion Models

Given a trained score function from (1) such that $s_{\theta}\approx\nabla_{\bm{\tau}^{i}}\log p_{i}\left(\bm{\tau}^{i}\right)$, a diffusion model denoises samples according to the distribution $p_{\theta}\left(\bm{\tau}^{i-1}|\bm{\tau}^{i}\right)=\mathcal{N}\left(\frac{1}{\sqrt{\alpha_{i}}}\left(\bm{\tau}^{i}+\left(1-\alpha_{i}\right)s_{\theta}\left(\bm{\tau}^{i},i\right)\right),\left(1-\alpha_{i}\right)\bm{I}\right)$ starting from a Gaussian prior $\bm{\tau}^{N}\sim\mathcal{N}\left(\bm{0},\bm{I}\right)$. For example, Diffuser [1] samples a trajectory $\bm{\tau}^{0}$ from a diffusion model, which an agent then executes. However, this original sampling process is unable to control detailed properties of generated context.

Here, we are interested in sampling trajectories that satisfy both the final goal and the specified instructions encoded as an $\textnormal{LTL}_{f}$ formula $\varphi$ that is provided during deployment. In other words, we aim to sample trajectories under an objective function $\mathcal{J}_{\varphi}\left(\bm{\tau}^{0}|g\right)$. For example, $\mathcal{J}$ can have a high value if the events induced by $\bm{\tau}^{0}$ satisfy $\varphi$ and have a low value otherwise.

Formally, given a set of atomic propositions $\mathcal{P}$, the assignments for $\bm{\tau}$ are given by a labeling function $L:\mathcal{S}\times\mathcal{A}\rightarrow 2^{\lvert\mathcal{P}\rvert}$, where each timestep of $\bm{\tau}$ induces an assignment to the propositional symbols in $\mathcal{P}$. For example, in navigation task, $\mathcal{P}$ can represent multiple regions to avoid and $L$ is a function indicating whether $(\mathbf{s}_{t},\mathbf{a}_{t})$ in $\bm{\tau}$ are inside these regions or not. We will slightly abuse notation and write $\bm{\tau}\models\varphi$ to indicate that $\bm{\tau}$’s assignments $\bm{\sigma}$ satisfy a $\textnormal{LTL}_{f}$ formula.

We aim to sample from the posterior $p_{0}\left(\bm{\tau}^{0}|\bm{\tau}^{0}\models\varphi,g\right)$ with the diffusion model as the prior. In this work, the constraint of goal state conditioning $g$ can be either achieved by inpainting (similarly on the start state $\mathbf{s}_{0}$) or implicitly encoded in the dataset, which does not require separate modeling in conditional sampling. Therefore, we mainly target the posterior $p_{0}\left(\bm{\tau}^{0}|\bm{\tau}^{0}\models\varphi\right)$. Let $y_{0}$ denote a binary random variable indicating the likelihood of $\bm{\tau}^{0}$ satisfying $\varphi$. Hence, the denoising process requires a score function conditioned on $y_{0}=1$ and by Bayes’ rule: $\nabla_{\bm{\tau}^{i}}\log p_{i}\left(\bm{\tau}^{i}|y_{0}=1\right)=\nabla_{\bm{\tau}^{i}}\log p_{i}\left(\bm{\tau}^{i}\right)+\nabla_{\bm{\tau}^{i}}\log p_{i}\left(y_{0}=1|\bm{\tau}^{i}\right)$. The first term on the right-hand side has been learned by a neural network in diffusion. However, the latter term requires an integration over all possible values of $\bm{\tau}^{0}$: $p_{i}\left(y_{0}|\bm{\tau}^{i}\right)=\int p\left(y_{0}|\bm{\tau}^{0}\right)p_{i}\left(\bm{\tau}^{0}|\bm{\tau}^{i}\right)\mathop{}\!\mathrm{d}\bm{\tau}^{0}$. We consider the plug-and-play conditional generation setting and approximating this integration with sample estimation [13], e.g., point estimation with $p_{i}\left(y_{0}|\bm{\tau}^{i}\right)\approx p\left(y_{0}|\hat{\bm{\tau}}^{0}\right)$ where the noiseless trajectory $\hat{\bm{\tau}}^{0}$ is estimated via Tweedie’s formula [14] $\hat{\bm{\tau}}^{0}=\frac{1}{\sqrt{\bar{\alpha}_{i}}}\left(\bm{\tau}^{i}+\left(1-\bar{\alpha}_{i}\right)\nabla_{\bm{\tau}^{i}}\log p_{i}\left(\bm{\tau}^{i}\right)\right)$. We model the likelihood term as $p\left(y|\bm{\tau}\right)=Ze^{\mathds{1}\left[y=\mathds{1}\left[\bm{\tau}\models\varphi\right]\right]}$, where $Z$ is a normalizing constant and $\mathds{1}\left[\cdot\right]$ is the indicator function. Putting the above elements together, the conditional score function can be computed as follows,

$$\begin{split}\nabla_{\bm{\tau}^{i}}\log p_{i}\left(y_{0}|\bm{\tau}^{i}\right)&\approx\nabla_{\bm{\tau}^{i}}\log p\left(y_{0}|\hat{\bm{\tau}}^{0}\right)\\ &=\nabla_{\bm{\tau}^{i}}\log\left(Ze^{\mathds{1}\left[y=\mathds{1}\left[\hat{\bm{\tau}}^{0}\models\varphi\right]\right]}\right)\\ &=\nabla_{\bm{\tau}^{i}}\mathds{1}\left[y=\mathds{1}\left[\hat{\bm{\tau}}^{0}\models\varphi\right]\right].\end{split}$$ (2)

Unfortunately, both the indicator function and the satisfaction evaluation (performed by techniques like model checking with finite automata) are non-differentiable — this prohibits application in the gradient-based sampling process of diffusion models. We address this problem in the next subsection.

III-B Differentiable Evaluation of $\textnormal{LTL}_{f}$

Our key approach is to “soften” the satisfaction evaluation. To make the evaluation differentiable, we modify our formula evaluator to output positive real values if a trajectory satisfies the $\textnormal{LTL}_{f}$ formula, and negative real values otherwise. Instead of using a binary labeling functions, we assume real valued assignments of atomic propositions can be obtained through computation on the generated trajectories. As a specific example, consider a navigation task where a robot has to avoid obstacles; one can determine the assignment for proposition $p$ at timestep $t$ using the Euclidean distance between $\mathbf{s}_{t}$ and the centers of the region $\mathbf{c}_{p}$ (assuming a circular shape). A positive value of $\ell(p,t)=r_{p}-\lVert\mathbf{s}_{t}-\mathbf{c}_{p}\rVert_{2}$, where $r_{p}$ is the radius of the circle, indicates a true assignment. Consequently, the labeling function for the entire trajectory $L:(\mathcal{S}\times\mathcal{A})^{T+1}\rightarrow\mathbb{R}^{\lvert\mathcal{P}\rvert\times(T+1)}$ is differentiable and the assignments are $\bm{\sigma}=L(\bm{\tau})$.

A binary version of $\bm{\sigma}$ through a sign function $\text{sgn}(\bm{\sigma})$ satisfies the definition of satisfy defined in Section II-B, but the sign operation breaks differentiability. As such, we employ a formula evaluator $f_{t}\left(\varphi,\bm{\sigma}_{t:T}\right):\Psi\times\mathbb{R}^{\lvert\mathcal{P}\rvert\times(T-t+1)}\rightarrow\mathbb{R}$ to check satisfaction [15], with positive values implying $\langle{\bm{\sigma},t}\rangle\models\varphi$, similar to signal temporal logic [16]. As such, $f$ is differentiable with the evaluation process defined as follows:

• $f_{t}(\mathsf{true}{},\bm{\sigma})=+\infty$

• $f_{t}(\mathsf{false}{},\bm{\sigma})=-\infty$

• $f_{t}(p,\bm{\sigma})=\bm{\sigma}_{t,p}$

• $f_{t}(\neg\varphi,\bm{\sigma})=-f_{t}(\varphi,\bm{\sigma})$

• $f_{t}(\varphi\wedge\psi,\bm{\sigma})=\min^{\gamma}\left\{f_{t}(\varphi,\bm{\sigma}),f_{t}(\psi,\bm{\sigma})\right\}$

• $f_{t}(\varphi\vee\psi,\bm{\sigma})=\max^{\gamma}\{f_{t}(\varphi,\bm{\sigma}),f_{t}(\psi,\bm{\sigma})\}$

• $f_{t}(\bigcirc\varphi,\bm{\sigma})=f_{t+1}(\varphi,\bm{\sigma})$

• $f_{t}(\Box\varphi,\bm{\sigma})=\min^{\gamma}\{f_{t:T}(\varphi,\bm{\sigma})\}$

• $f_{t}(\Diamond\varphi,\bm{\sigma})=\max^{\gamma}\{f_{t:T}(\varphi,\bm{\sigma})\}$

• $f_{t}(\varphi\operatorname{\mathsf{U}}{}\psi,\bm{\sigma})=\min^{\gamma}\{f_{t:k}(\varphi,\bm{\sigma}),f_{t}\left(\Diamond\psi,\bm{\sigma}\right)\}$, where $k\geq t$ is the smallest integer s.t. $f_{k}(\psi,\bm{\sigma})>0$

Note that the $\min$ and $\max$ functions are likewise “soft” to maintain differentiability. To reduce clutter, we have omitted the subscript $t$ when $t=0$. With these operations, $f$ maintains quantitative semantics, which preserves the relative values between different $\bm{\sigma}$ such that trajectories with larger margin to a satisfying assignment have larger values.

Given $L$ and $f$, we can obtain a differentiable score function by replacing the likelihood term in (2) with $p\left(y|\bm{\tau}\right)=Ze^{(2y-1)f\left(\varphi,L\left(\bm{\tau}\right)\right)}$. The conditional score is then

$$\begin{split}\nabla_{\bm{\tau}^{i}}\log p_{i}\left(y_{0}=1|\bm{\tau}^{i}\right)&\approx\nabla_{\bm{\tau}^{i}}\log\left(Ze^{f\left(\varphi,L\left(\hat{\bm{\tau}}^{0}\right)\right)}\right)\\ &=\nabla_{\bm{\tau}^{i}}f\left(\varphi,L\left(\hat{\bm{\tau}}^{0}\right)\right).\end{split}$$ (3)

This approximation of the conditional score function can be directly used with a pre-trained diffusion model to sample trajectories conditioned on an $\textnormal{LTL}_{f}$ formula. We call this method LTLDoG-S, since it performs the above posterior sampling in the reverse process (Algorithm 1). The gradient ascent step is controlled by a stepsize $\{\zeta_{i}\}_{i=1}^{N}$; in practice, the stepsize for each denoise step can be adaptive such that $\bm{\tau}^{i-1}$ remains valid according to the formula evaluator $f\left(\varphi,L\left(\hat{\bm{\tau}}^{0}\right)\right)$.

Algorithm 1 $\textnormal{LTL}_{f}$ Planning with Posterior Sampling

1:$\varphi$, $N$, $s_{\theta}$, $\{\zeta_{i}\}_{i=1}^{N}$

2:$\bm{\tau}^{N}\sim\mathcal{N}\left(\bm{0},\bm{I}\right)$

3:for $i=N-1$ to $0$ do

4:     $\hat{s}\leftarrow s_{\theta}\left(\bm{\tau}^{i},i\right)$

5:     $\hat{\bm{\tau}}^{0}\leftarrow\frac{1}{\sqrt{\bar{\alpha}_{i}}}\left(\bm{\tau}^{i}+\left(1-\bar{\alpha}_{i}\right)\hat{s}\right)$

6:     $\bm{\epsilon}\sim\mathcal{N}\left(\bm{0},\bm{I}\right)$

7:     $\bm{\tau}^{i-1}\leftarrow\frac{\sqrt{\alpha_{i}}\left(1-\bar{\alpha}_{i-1}\right)}{1-\bar{\alpha}_{i}}\bm{\tau}^{i}+\frac{\sqrt{\bar{\alpha}_{i-1}}(1-\alpha_{i})}{1-\bar{\alpha}_{i}}\hat{\bm{\tau}}^{0}+\sqrt{1-\alpha_{i}}\bm{\epsilon}$

8:     $\bm{\tau}^{i-1}\leftarrow\bm{\tau}^{i-1}+{\zeta_{i}}\nabla_{\bm{\tau}^{i}}f\left(\varphi,L\left(\hat{\bm{\tau}}^{0}\right)\right)$

9:end for

10:return $\hat{\bm{\tau}}^{0}$

III-C Classifier Guidance over $\textnormal{LTL}_{f}$

One limitation of LTLDoG-S is that it necessitates a known (and differentiable) expression for the formula evaluator $f\left(\varphi,L\left(\hat{\bm{\tau}}^{0}\right)\right)$. This requirement can be challenging to meet in scenarios where the truth assignments of propositions are uncertain, for example, when the ground-truth physical dynamics are unknown. Here, we circumvent this problem by employing classifier guidance using a trained formula evaluator.

We propose a variant of LTLDoG with regressor guidance, which we abbreviate as LTLDoG-R. Using an $\ell_{2}$ loss, we train a neural network to predict the satisfaction values from noisy trajectories conditioned on $\textnormal{LTL}_{f}$ instructions. In other words, our neural network approximates the conditional score function $\nabla_{\bm{\tau}^{i}}\log p_{i}\left(y_{0}=1|\bm{\tau}^{i}\right)$. Note that in contrast to learning a binary classifier for trajectory satisfaction, we apply the labeling function in Section III-B on the dataset and associate with each trajectory its objective value $\mathcal{J}_{\varphi}\left(\bm{\tau}^{0}\right)$. We conduct ablation study in Section V-D to show that using real values performs better than using binary labels.

To generalize over different $\textnormal{LTL}_{f}$, the neural network takes both noisy trajectory and $\textnormal{LTL}_{f}$ formula embedding as input. Multiple methods exist for embedding $\textnormal{LTL}_{f}$ formulae. For instance, one can use Graph Neural Networks (GNNs) [17, 18] to embed the tree representation of an $\textnormal{LTL}_{f}$ formula directly. Alternatively, the deterministic finite-state automaton (DFA) [19, 20] associated with the formulae can be embedded [21]. In this work we employ $\textnormal{LTL}_{f}$ tasks from [22] and embed the directed graph of an $\textnormal{LTL}_{f}$ formula using the Relational Graph Convolutional Network (R-GCN) [23], which can generalize to $\textnormal{LTL}_{f}$ formulae with same template structure. The model that approximates the score function $s_{\phi}(\phi,\bm{\tau}^{i},i)\approx\nabla_{\bm{\tau}^{i}}\log p_{i}\left(y_{0}|\bm{\tau}^{i}\right)$ after training can be plugged into the conditional reverse process using regressor guidance as summarized in Algorithm 2.

Algorithm 2 $\textnormal{LTL}_{f}$ Planning with Regressor Guidance

1:$\varphi$, $N$, $s_{\theta}$, $s_{\phi}$, $\{\zeta_{i}\}_{i=1}^{N}$

2:$\bm{\tau}^{N}\sim\mathcal{N}\left(\bm{0},\bm{I}\right)$

3:for $i=N-1$ to $0$ do

4:     $\bm{\epsilon}\sim\mathcal{N}\left(\bm{0},\bm{I}\right)$

5:     $\bm{\tau}^{i-1}\leftarrow\frac{1}{\sqrt{\alpha_{i}}}\left(\bm{\tau}^{i}+(1-\alpha_{i})s_{\theta}\left(\bm{\tau}^{i},i\right)\right)+\sqrt{1-\alpha_{i}}\bm{\epsilon}$

6:     $\bm{\tau}^{i-1}\leftarrow\bm{\tau}^{i-1}+\zeta_{i}s_{\phi}(\phi,\bm{\tau}^{i},i)$

7:end for

8:return $\hat{\bm{\tau}}^{0}$

IV Related Work

LTLDoG builds upon prior work in diffusion-based planning and symbolic reasoning using $\textnormal{LTL}_{f}$ for robotics. In the following, we give a brief overview of related work.

Learning and Planning under $\textnormal{LTL}_{f}$. As an expressive language for specifying high-level planning requirements [24, 25, 26], $\textnormal{LTL}_{f}$ has been extensively used in various robotic tasks to express temporally extended goals [27, 28]. These methods usually require the information about the environment’s dynamics, e.g., a model or an abstraction, to effectively plan under a given formula. Reinforcement learning agent learn in a model-free way under $\textnormal{LTL}_{f}$ objectives or constraints [29, 30, 31] with the ability to generalize over different formulae [22]. However, these methods operate agent in an online manner via trial and error, which can lead to expensive or even unsafe interactions.

Planning and Policy Learning with Diffusion. Recent diffusion-based planning methods are flexible that only rely on offline datasets without access or interaction to environments. They have been successfully applied to long-horizon planning problems by generating states or actions for control [1, 3, 2], but not tasks with test-time temporal requirements. Recent work has looked into safety critical tasks e.g., the aforementioned CBF-based methods [7, 8]. As discussed above, these methods were designed for static safety criteria, and the lack the ability to satisfy temporally extended behaviors. Our work inherits the advantages of diffusion based methods and can fulfill $\textnormal{LTL}_{f}$ requirements.

Inverse Problems in Diffusion. Our proposed method formulates conditional measurement under $\textnormal{LTL}_{f}$ with differentiable loss function using the unnormalized likelihood [13, 32] for posterior sampling and can guide the sampling process in a plug-and-play fashion. The most popular methods to guide diffusion models during inference is classifier guidance [33] and classifier-free guidance [34]. However, these methods cannot be applied in a plug-and-play fashion for new conditioning factors. Our work is also related to inverse task that infers a posterior distribution over data given a measurement. Inverse methods [35, 36, 37, 38] do not require training a conditional model in diffusion and can directly leverage pre-trained neural networks.

(a) Studio-like Lab.

(b) Office room.

(c) Unitree Go2.

Figure 2: Real world environments for quadruped robot navigation.

V Experiments

TABLE I: Performance on Avoidance Tasks in Maze2d

Method\Perf.	Satisfaction rate1 ($\%$) $\uparrow$		Reward (UnCon) 2 $\uparrow$
	Planning	Rollout
Diffuser	9.5$\pm$3.1	11.0$\pm$1.7	142.2$\pm$5.1
SafeDiffuser	99.4$\pm$0.9	12.3$\pm$3.5	135.8$\pm$5.2
LTLDoG-S	99.0$\pm$0.8	73.0$\pm$3.0	97.3$\pm$2.8
LTLDoG-R	98.8$\pm$0.8	92.0$\pm$1.4	127.1$\pm$5.1

• Mean and standard deviation calculated from $10$ groups of tests. Each test contains $100$ trials, where a trajectory is labeled as either satisfied or not satisfied in each trial. Best result during rollout is highlighted. Same for other tables of Maze2d.

• Unconstrained rewards do not take unsafe penalties into account.

TABLE II: Results of Avoidance Task in PushT

Method\Perf.	Satisfaction rate($\%$) $\uparrow$	Overlap Score1 $\uparrow$
Diffusion Policy	34.8$\pm$18.0	0.941$\pm$0.0584
LTLDoG-S	85.6$\pm$13.1	0.890$\pm$0.0647
LTLDoG-R	85.6$\pm$12.5	0.842$\pm$0.0985

• measures the final overlap of the T block and target area (min: $0.0$, max: $1.0$).

(a) Diffuser.

(b) SafeDiffuser.

(c) LTLDoG-S.

Figure 3: Examples of safe planning in Maze2d-Large. There are three unsafe blocks (red squares, labeled $p_{L}$, $p_{M}$, $p_{R}$ from left to right) that need to be avoided during navigation to the goal (shaded circle). The $\textnormal{LTL}_{f}$ constraint for this task is $\varphi=\Box\neg(p_{L}\wedge p_{M}\wedge p_{R})$. (a) Trajectories from Diffuser ignore safety and can violate the specified constraints. (b) SafeDiffuser produces discontinuous trajectories. (c) Our LTLDoG is able to plan trajectories that detours around the obstacles to successfully arrive at the goal.

(a) PushT environment.

(b) Diffusion Policy.

(c) LTLDoG-S.

Figure 4: Results of safe control in PushT. (a) A robot arm’s end effector (circles filled in blue) should manipulate the T block (gray) to a goal pose (green), and avoid entering unsafe regions (hollow circles marked with $pX$), specified by an $\textnormal{LTL}_{f}$ formula (text in black). In this example, the $\textnormal{LTL}_{f}$ specifies the end effector should never enter regions $p1$ (purple) and $p3$ (cyan). (b) The actions generated and executed by Diffusion Policy do not satisfy the $\textnormal{LTL}_{f}$ formula. (c) In contrast, LTLDoG-S guides the diffusion to avoid $p1$ and $p3$, yet still completes the manipulation task.

Our experiments focus on testing LTLDoG’s ability to handle static and temporal safety constraints. We first briefly describe the simulated and real environments, and baseline data-driven methods. Then we report empirical results on benchmark environments and demonstrate LTLDoG’s applicability in real world tasks through a case study on a quadruped robot (Fig. 2). We conclude with a brief ablation study and analysis. Due to space restrictions, we focus on conveying our main results. More details of the environments, implementations and analysis can be found in the appendix.

V-A Experimental Setup

Environments. We evaluate methods in two simulation benchmark environments (Maze2d [1] and PushT [3]) and demonstrate in two real indoor rooms. Maze2d (Fig. 3 and 5) presents challenging long-horizon navigation tasks, where state-of-the-art offline algorithms fail to learn a reliable goal-reaching policy. The atomic propositions are determined by the occurrence of events when the agent is inside key regions in the maze (for avoidance and visitation). The PushT task (Fig. 4) requires manipulation of a T block through interaction with a controllable mover. In our experiments, the mover is constrained to visit specific regions and avoid others. Our real-world experiments involve two indoor environments: a lab designed to mimic a studio apartment, and an office environment (Fig. 2).

Compared methods. Our work involves trajectory generation by learning from an offline dataset and as such, we compare against data-driven planning methods. Diffuser and Diffusion Policy are state-of-the-art methods for sampling viable plans but without any guarantees over external constraints. To evaluate how well LTLDoG enforces safety specifications, we compare with SafeDiffuser, a safe planning diffusion model using CBFs. However, note that SafeDiffuser cannot handle temporal constraints or instructions; to our knowledge, our work is the first data-driven method to handle both static and temporal constraints. As such, there is no direct comparison baseline. We analyze the differences between the two variants of our method, LTLDoG-S and LTLDoG-R.

V-B Comparative Analysis of Methods

Can LTLDoG achieve safe planning for static constraints? Our results indicate that yes, LTLDoG is better able to generate trajectories that satisfy given region-avoidance constraints relative to existing methods. In both the Maze2D and PushT benchmarks, LTLDoG achieves high success rates (in both planning and rollout), without severely compromising reward accumulation (Tables I and  II). Lower total rewards are expected since safe paths are typically longer and rollout scores are generally lower as the low-level controller may not exactly follow the diffused trajectory. Qualitatively, Fig. 3 shows that LTLDoG is able to generate safe trajectories in Maze2D, whilst SafeDiffuser is limited to “local” deviations and fails to find paths that detour around unsafe regions. Results in PushT are consistent with Maze2D; Fig. 4 shows that LTLDoG performs the orientation task without entering unsafe regions.

(a) Regions for evaluation of atomic propositions in $\mathcal{P}$.

(b) Diffuser.

(c) Ours (LTLDoG-S).

Figure 5: Temporal Constraints in Maze2D. (a) Each maze has 6 non-overlapping regions. Agents are requested to visit some of these blocks under different temporally-extended orders. (b) and (c) show generated trajectories under $\varphi=\neg p_{3}\operatorname{\mathsf{U}}{}(p_{5}\wedge(\neg p_{2}\operatorname{\mathsf{U}}{}p_{0}))$. Our method can satisfy $\neg$ propositions (red zones) before reaching the green regions.

TABLE III: Performance on Different $\textnormal{LTL}_{f}$s in Maze2d.

Environment	Method\Performance		Training $\textnormal{LTL}_{f}$s			Testing $\textnormal{LTL}_{f}$s
			Satisfaction rate ($\%$) $\uparrow$		Reward (UnCon) $\uparrow$	Satisfaction rate ($\%$) $\uparrow$		Reward (UnCon) $\uparrow$
			Planning	Rollout		Planning	Rollout
U-Maze (Horizon 256)	Diffuser		31.1$\pm$0.5	31.0$\pm$0.5	33.5$\pm$2.7	33.9$\pm$0.5	34.1$\pm$0.6	35.6$\pm$0.3
	LTLDoG-S		83.8$\pm$0.2	57.6$\pm$1.3	31.3$\pm$1.2	82.7$\pm$0.3	56.6$\pm$0.9	32.8$\pm$0.5
	LTLDoG-R		56.3$\pm$0.4	51.3$\pm$0.9	31.5$\pm$0.2	57.7$\pm$0.4	52.1$\pm$0.3	32.3$\pm$0.7
Medium (Horizon 384)	Diffuser		15.0$\pm$0.7	13.4$\pm$0.6	84.8$\pm$0.3	11.6$\pm$1.4	10.1$\pm$1.2	84.8$\pm$0.5
	LTLDoG-S		77.9$\pm$5.7	31.8$\pm$2.6	53.1$\pm$5.2	68.4$\pm$6.7	28.7$\pm$3.5	50.5$\pm$4.7
	LTLDoG-R		51.8$\pm$1.8	39.5$\pm$1.6	57.3$\pm$0.2	43.3$\pm$4.4	30.6$\pm$1.9	57.7$\pm$0.1
Large (Horizon 512)	Diffuser		13.5$\pm$0.4	12.8$\pm$0.1	76.3$\pm$0.1	11.6$\pm$2.3	11.5$\pm$1.7	77.8$\pm$3.9
	LTLDoG-S		73.8$\pm$2.4	32.6$\pm$1.4	42.3$\pm$5.0	66.6$\pm$2.7	24.9$\pm$1.7	40.9$\pm$4.4
	LTLDoG-R		66.9$\pm$0.6	47.4$\pm$0.8	54.6$\pm$1.3	57.5$\pm$2.3	39.0$\pm$2.9	54.5$\pm$3.9

TABLE IV: Generalization to Different $\textnormal{LTL}_{f}$ formulae in PushT

Method\Performance	$\textnormal{LTL}_{f}$ Set	Satisf. rate ($\%$) $\uparrow$	Score $\uparrow$
Diffusion Policy	Training	22.9$\pm$8.0	0.354$\pm$0.153
	Test	30.7$\pm$13.9	0.371$\pm$0.177
LTLDoG-S	Training	28.2$\pm$8.33	0.290$\pm$0.115
	Test	43.0$\pm$17.0	0.299$\pm$0.145
LTLDoG-R	Training	69.3$\pm$9.90	0.292$\pm$0.121
	Test	66.0$\pm$20.8	0.340$\pm$0.168

Can LTLDoG satisfy static and temporal constraints, and generalize to novel $\textnormal{LTL}_{f}$ formulae? Tables III and IV show the performance of the compared methods on both training and test LTLs in Maze2d and PushT environments. To elaborate, we follow the $\textnormal{LTL}_{f}$ specifications in [22] and adopt the Until sampler to generate random $\textnormal{LTL}_{f}$s ($200$ for Maze2d and $36$ for PushT) that contain different visitation sequences and unsafe regions. The training set has $80\%$ of all $\textnormal{LTL}_{f}$s and the rests are used as test set. Atomic propositions consist of 6 regions in Maze2d (Fig. 5(a), i.e., $p_{0},p_{1},\ldots,p_{5}$) and 4 regions in PushT (Fig. 4(a)).

Results reveal that both LTLDoG-S and LTLDoG-R achieve significantly higher success rates than Diffuser/Diffusion Policy. The baselines have a non-zero performance as some generated $\textnormal{LTL}_{f}$ formulae are trivial to satisfy at some start locations. Also note that some specifications may be impossible to satisfy given the physical locations of the agent, walls, and propositional regions in the maze. In these cases, planned trajectories will violate constraints during rollout, causing a drop in the satisfaction rate. Fig. 5 shows examples of trajectories generated in Maze2d.

(a) Lab map and dataset coverage.

(b) Diffuser.

(c) Ours.

(d) Office map and dataset coverage.

(e) Diffuser.

(f) Ours.

Figure 6: Results in real world rooms. The instructed $\textnormal{LTL}_{f}$ is $\varphi=\neg\text{Passageway}\operatorname{\mathsf{U}}{}\text{Kitchen}$ for lab (first row) and $\varphi=\Diamond\left(\text{Door}\wedge\Diamond\text{Seat}\right)$ for office (second row). In the lab task the robot has to first unload in the kitchen area after entering the door before being allowed to go to the goal. When loaded, the robot is prohibited from going through the narrow passageway where people sit on the side. In the office task, the robot has to first visit the seat, followed by the door, before the goal.

TABLE V: Results of achieving goals and $\textnormal{LTL}_{f}$ in real-world tasks.

Environment	Method\Performance		Satisfaction rate ($\%$) $\uparrow$
			Goal	$\textnormal{LTL}_{f}$
Lab	Diffuser		100.00	0.00
	LTLDoG-S		91.67	91.67
Office	Diffuser		100.00	0.00
	LTLDoG-S		95.83	95.83

V-C Real Robot Case Study.

We show that LTLDoG can plan for a robot dog (Unitree Go2 Edu) given $\textnormal{LTL}_{f}$ instructions in two real-world navigation environments — a lab that mimics a studio apartment and an office room. Training of diffusion models was performed in simulation using Gazebo and ROS1 using LIDAR scanned maps (Fig. 6(a) and 6(d)). Note that the training trajectories do not require running an oracle policy to satisfy many different $\textnormal{LTL}_{f}$ formulae; we simply made the robot navigate to randomly sampled goals using the global planner and TEB Local Planner [39] from the ROS1 Navigation stack (Fig. 6(a) and 6(d)).

To test on potential constraints, we queried GPT-4 for $\textnormal{LTL}_{f}$ formulae representing meaningful robotic tasks including obstacle avoidance and sequential navigation. We used 4 kinds of $\textnormal{LTL}_{f}$s from the generated results, i.e., $\Box\neg p_{0}$, $\Diamond p_{0}\wedge\Diamond p_{1}$, $\Diamond\left(p_{0}\wedge\Diamond p_{1}\right)$ and $\neg p_{1}\operatorname{\mathsf{U}}{}p_{0}$. The first $\textnormal{LTL}_{f}$ corresponds to an obstacle avoidance task where the robot should never visit a specific region. For example, $\Box\neg\text{Seat}$ means the robot should never enter the Seat region (Fig 6(d)). The remaining three $\textnormal{LTL}_{f}$ formulae represent: 1) visiting all regions at least once; 2) visiting regions in a specific sequence; 3) avoiding a specific region until another has been visited. Some example regions designed in our real environments and start/goal locations are shown in Fig 6.

In total, $96$ trials were executed on the real robot, involving $12$ trajectories ($6$ for baseline Diffuser and $6$ for our method LTLDoG-S) with varying regions and different $\textnormal{LTL}_{f}$ formulae in each room. The overall satisfaction rate of all raw generated trajectories in simulation is $85.8\pm 14.0\%$ (c.f. baseline $2.9\pm 5.4\%$). For each specific formula, a sample was selected for real-world execution based on their feasibility. The results in TABLE V show that LTLDoG has a high satisfaction rate compared to Diffuser; Fig. 6(c) and 6(f) illustrate trajectories for qualitative comparison.

V-D Ablation Study and Analysis

TABLE VI: Ablation Study – Binary Classifier Guidance

Method\Performance	LTL Set	Satisfaction rate ($\%$) $\uparrow$		Reward (UnCon) $\uparrow$
		Planning	Rollout
Diffuser	Training	31.1$\pm$0.5	31.0$\pm$0.5	32.5$\pm$2.7
	Test	33.9$\pm$0.5	34.1$\pm$0.6	35.6$\pm$0.3
Classifier guidance1	Training	41.1$\pm$0.6	40.6$\pm$0.7	33.9$\pm$0.9
	Test	40.0$\pm$0.8	41.8$\pm$1.1	35.3$\pm$0.3
LTLDoG-R	Training	56.3$\pm$0.4	51.3$\pm$0.9	31.5$\pm$0.2
	Test	57.7$\pm$0.4	52.1$\pm$0.3	32.3$\pm$0.7

• The classifier guidance method only leverages binary labels for LTL satisfaction checking in Maze2d U-Maze.

Ablation study. Unlike classifier guidance, where each trajectory is labelled as satisfy or not satisfy, we leverage the continuous values from our formula evaluator (as described in Section III-C) and train a regressor guidance network. Although classifier guidance improves over the vanilla Diffuser, it achieves lower performance than LTLDoG-R ( Table VI). This comparison supports the notion that soft labels improves the guidance neural network; we posit using real values provides richer information in terms of how well the trajectory satisfies a given $\textnormal{LTL}_{f}$ formula.

Analysis on horizon. Fig. 7 demonstrates that planning with a longer horizon leads to improved performance in terms of $\textnormal{LTL}_{f}$ satisfaction. This improvement is attributed to the fact that $\textnormal{LTL}_{f}$ instructions often require a longer sequence of steps in a path compared to mere goal navigation, e.g., visiting a specific region before reaching the goal.

(a) Planning.

(b) Rollout.

Figure 7: Performance in Maze2d U-Maze with different lengths of trajectory. DM stands for Diffuser and DoG-S/DoG-R are our methods.

VI Conclusion, Discussion and Future Work

In this work, we presented LTLDoG, an approach towards generating safe trajectories that comply with $\textnormal{LTL}_{f}$ specifications at test time. Within our overall scheme, we presented two methods: LTLDoG-S guides the sampling process under any $\textnormal{LTL}_{f}$ formula while LTLDoG-R uses a trained model that we show generalizes to new formulae with similar structure. To our knowledge, this work is the first that successfully incorporates model checking using a formal language with diffusion models for safe planning. Notably, LTLDoG does not require data collection for each potential $\textnormal{LTL}_{f}$ instruction; rather, we control the sampling process during diffusion using “soft” model checking to generate new trajectories using existing information provided by the training dataset.

Limitations and Future Work. LTLDoG is a step towards trustworthy trajectory planning using generative models. There are several areas where LTLDoG can be improved. Similar to other diffusion models, LTLDoG is generally unable to generate realistic trajectories when the context (environment/goal) is far from the training distribution. As such, the dataset should preferably contain trajectories with a variety of complex behaviors that can potentially satisfy different test-time $\textnormal{LTL}_{f}$ formulae. It would be interesting to develop methods to ensure the sampling process of LTLDoG-S adheres to the data manifold. Additionally, LTLDoG plans in an open-loop fashion, where the entire trajectory is generated conditioned on the constraints. This is mainly because the evaluation of an $\textnormal{LTL}_{f}$ formula depends on the entire trajectory. We plan to further explore planning using receding horizon control using partial evaluations on an incomplete trajectory. Finally, diffusion models usually require significant amounts of training data and many diffusion steps during inference. Recent work on interpolant diffusion methods [40] leverages source policies to reduce data and computation costs. We aim to explore how integrating this approach with conditional sampling using $\textnormal{LTL}_{f}$ can enhance the generation of safe trajectories.

-A Data

-A1 Maze2d

The Maze2d environment is part of a well-known open-source benchmark for offline reinforcement learning — D4RL [41]. Datasets of trajectories are publicly available in the D4RL library. Each of the trajectories within is one continuous trajectory of an agent navigating to random goals without interruption. See Fig. 8 for an overview of the environments. We utilize the original dataset in D4RL as is to train the baseline Diffuser, which also serves as the diffusion model of LTLDoG.

(a) U-Maze.

(b) Medium.

(c) Large.

Figure 8: Maze2d environments. The agent is tasked to move force-actuated ball to goals (red circle) from random start locations (green ball). There are three kinds of maze configurations with increasing difficulties and all of them are long-horizon path planning tasks with continuous states and actions.

-A2 PushT

PushT is a robot manipulation benchmark adopted in Diffusion Policy [3] to showcase diffusion models’ planning capabilities. A dataset of expert policies for pushing a T block to the same goal position starting from random poses are publicly available at [3]. Though Diffusion Policy achieves good performance on the original manipulation task, trained with around only 200 expert demonstrations (i.e., 206 episodes), we find that these trajectories lack the modalities or abundant behaviors that potentially or partially satisfy different $\textnormal{LTL}_{f}$ constraints. As such, expanding the original dataset is necessary for our method to work.

Data Augmentation Given that obtaining extra expert demonstrations is inefficient, we performed common data augmentation techniques to expand the original dataset. Concretely, we first employed simple geometric augmentation, only rotation and flipping, for each trajectory in the original dataset, assuming basic physical symmetry in this simulation environment. Then, we used a pretrained Diffusion Policy to push the T block to the goal pose, aligning with the original dataset in which the goal of the manipulation task is implicitly encoded in trajectories. In this way, we augmented the original set from 206 trajectories to 6686 trajectories, and elongated the average episode length from 125 to 289 steps. The augmented dataset is publicly available online.

In this task, the baseline (Diffusion Policy) and our methods share the same diffusion model backbone, which is trained over the augmented dataset.

-A3 Real Robot Scenarios

Data for both of the real-world scenarios are collected in simulated environments. For the office we collected 4 long continuous trajectories in the simulation of step size of 11,000, 107,000, 60,000 and 130,000. Similarly for the lab map we collected 4 trajectories of steps 11,000, 19,000, 110,000 and 120,000. First, we built by SLAM the maps of the two environments, a studio-like lab and an office room, and the maps are used to create two simulation scenarios. For creating the maps, we teleoperated a quadruped robot (Unitree Go2) mounted with LiDAR in each of these environments. We used the slam_toolbox along with Nav2 for generating the maps in both environments. We created simple 3D world for each of the maps for simulation in gazebo. After that, we enabled a Go2 robot simulated in Gazebo and driven by ROS Navigation Stack to navigate to random goal positions sampled in the maps for data collection, creating long trajectories in a similar fashion to Maze2d.

-A4 LTL Satisfaction Values

The soft labels (values indicating how well the $\textnormal{LTL}_{f}$s are satisfied) for training the regressor guide (LTLDoG-R) are calculated with an adapted version of the DTL toolbox [15] and saved before each training session, and can be reused if possible, since each training session may sample and cut different segments of the trajectories in datasets based on different training configurations. Values calculation for benchmarking test results are performed online simultaneously with all tests.

-B $\textnormal{LTL}_{f}$ Generation

To generate practical $\textnormal{LTL}_{f}$s meaningful for real robotic tasks, we query GPT-4 via prompting with some constraints on $\textnormal{LTL}_{f}$s. The prompt and the response from ChatGPT is listed in TABLE VII. In the real experiments, we tested the first four returned $\textnormal{LTL}_{f}$s as these are the most representative $\textnormal{LTL}_{f}$s among them and are also commonly used in robot learning literature [30, 42, 22]. Some of the other generated $\textnormal{LTL}_{f}$s shown in this table are either not logically correct or not meaningful for robotic tasks. For instance, the $5$th $\textnormal{LTL}_{f}$ has a similar meaning of sequential visiting as the $2$nd $\textnormal{LTL}_{f}$. The imply operator does not enforce any constraints if the condition is not met (e.g., the condition in the $6$th $\textnormal{LTL}_{f}$ can never be satisfied for disjoint regions). The $7$th, $8$th and $9$th $\textnormal{LTL}_{f}$s are not correct with their corresponding descriptions. The last $\textnormal{LTL}_{f}$ can be considered as a combination of the $2$nd and $3$rd $\textnormal{LTL}_{f}$s. More complex $\textnormal{LTL}_{f}$s and combinations of them can be derived based on different visitation and avoidance patterns [43]. We plan to further explore these temporal specifications in environments like Robomimic, and adapting our method to complex combinations of $\textnormal{LTL}_{f}$s in future work.

TABLE VII: Prompts and Responses from ChatGPT (GPT-4)¹ for $\textnormal{LTL}_{f}$ Generation.

Prompt		In a robot navigation task, the robot is required to visit four key regions (referred to as p_0, p_1, p_2 and p_3) in specific temporal orders. The constraints on the robot’s trajectory concerning the visitation (or non-visitation) of these regions can be specified using finite Linear Temporal Logic (LTLf). Please generate several LTLf formulas (e.g., 10) that are realistic in real-world environments and tasks involving robot navigation. The details of the task and additional requirements are as follows: 1. The LTLf operators that can be used are: $\neg$ (Negate), $\wedge$ (And), $\vee$ (Or), F (Eventually), G (Always), U (Until). Avoid using other LTL operators. Also, avoid using X (Next) since the number of timesteps needed to move from one region to another is unknown. 2. The atomic propositions are determined by whether the robot is inside p_0, p_1, p_2 and p_3. Therefore, the labelling function returns assignments with a 4-dimensional 0-1 vector at each timestep of the robot’s trajectory. 3. Only one proposition can be true at each timestep because these regions are disjoint. The robot cannot be inside any two regions simultaneously. An assignment at a timestep can be a zero vector (0, 0, 0, 0), indicating that the robot is not currently in any of these regions. This occurs when the robot is initially not in these regions and all atomic propositions are False. 4. The robot cannot move directly from one region to another as the regions might not be connected. Therefore, LTLf formulas such as p_0 U p_1 are not valid since both propositions will be False when the robot is not inside these two regions. 5. The robot’s trajectory is finite, so the robot cannot repeatedly visit different regions. 6. The LTLf formulas are supposed to apply from the first timestep of the assignment. 7. When generating an LTLf formula, please analyze the generated formula first to ensure that it is valid and that there exist assignments that can satisfy it.
ChatGPT		To create realistic LTLf formulas for a robot navigation task considering the specified constraints and operators, we need to structure these formulas in a way that reflects possible navigation scenarios. Here are ten LTLf formulas that align with the given requirements: 1. Visit all regions at least once: • $F(p_{0})\wedge F(p_{1})\wedge F(p_{2})\wedge F(p_{3})$ This formula states that eventually, the robot will visit each of the four regions at least once during its trajectory. 2. Visit regions in a specific sequence: • $F(p_{0}\wedge F(p_{1}\wedge F(p_{2}\wedge F(p_{3}))))$ This formula requires the robot to visit $p_{0}$, then $p_{1}$, followed by $p_{2}$, and finally $p_{3}$, in that order. 3. Avoid a specific region until another has been visited: • $\neg p_{1}\operatorname{\mathsf{U}}{}p_{0}$ Here, the robot must avoid $p_{1}$ until $p_{0}$ has been visited. 4. Never visit a specific region: • $G(\neg p_{3})$ This formula specifies that the robot should never enter region $p_{3}$. 5. Visit one region only after visiting two others: • $F(p_{0}\wedge F(p_{1}))\operatorname{\Rightarrow}{}F(p_{2})$ The robot can visit $p_{2}$ only after it has visited $p_{0}$ and then $p_{1}$. 6. Ensure two regions are visited before a third: • $F(p_{0}\wedge p_{1})\operatorname{\Rightarrow}{}F(p_{2})$ This formula ensures that $p_{2}$ and $p_{1}$ are visited (in no particular order) before visiting $p_{2}$: 7. Visit at least three regions: • $F(p_{0})\vee F(p_{1})\vee F(p_{2})\vee F(p_{3})$ This ensures the robot visits at least three out of the four regions. It is a less strict requirement, allowing for flexibility in navigation. 8. Visit a specific region last: • $F(p_{0}\wedge F(p_{1}\wedge F(p_{2})))\wedge G(\neg p_{3}\operatorname{\mathsf{U}}{}(p_{0}\wedge F(p_{1}\wedge F(p_{2}))))$ The robot must visit $p_{3}$ last, after visiting $p_{0}$, $p_{1}$, and $p_{2}$ in any order. 9. Avoid a region until all other regions have been visited: • $\neg p_{3}\operatorname{\mathsf{U}}{}(F(p_{0})\wedge F(p_{1})\wedge F(p_{2}))$ The robot should avoid $p_{3}$ until it has visited $p_{0}$, $p_{1}$, and $p_{2}$ in any order. 10. Visit a pair of regions, avoiding others until then: • $\neg(p_{2}\vee p_{3})\operatorname{\mathsf{U}}{}(p_{0}\wedge F(p_{1}))$ The robot must avoid $p_{2}$ and $p_{3}$ until it has visited $p_{0}$ and then $p_{1}$: Each of these formulas considers the finite nature of the robot’s trajectory, the impossibility of being in multiple regions simultaneously, and the requirement that the robot may need to travel through unspecified areas (not within $p_{0}$, $p_{1}$, $p_{2}$, or $p_{3}$) to move from one region to another. The formulas are structured to reflect realistic navigation tasks that could be encountered in real-world robot navigation scenarios, such as search and rescue, surveillance, or delivery tasks within a specified area.

• https://chat.openai.com/share/43670685-3cb9-4496-ad3e-1ee70cd358b8.

-C Additional Illustrations

More illustrations and comparisons against the baseline for the obstacle avoidance task in Maze2d environment are in Fig. 9.

(a) Trajectories from Diffuser.

(b) Trajectories from SafeDiffuser.

(c) Trajectories from LTLDoG-S.

(d) Trajectories from LTLDoG-R.

Figure 9: Results of safe planning in Maze2d-Large with unsafe regions. There are three unsafe blocks (red squares, labeled $p_{L}$, $p_{M}$, $p_{R}$ from left to right) that need to be avoided during navigation to the goal (shaded circle). The $\textnormal{LTL}_{f}$ constraint for this task is $\varphi=\Box\neg(p_{L}\wedge p_{M}\wedge p_{R})$. (a) Trajectories from Diffuser ignore safety and can violate the specified constraints. (b) SafeDiffuser produces discontinuous trajectories (the constraints here are slightly reshaped to accommodate the CBF function). (c, d) Our LTLDoG is able to plan trajectories that detours around the obstacles to successfully arrive at the goal.

TABLE VIII: Results of generated $\textnormal{LTL}_{f}$s in simulation rooms.

Task	Method\Performance	Success rate ($\%$) $\uparrow$
		Lab	Office
Obstacle avoidance	Diffuser	11.7$\pm$3.7	0.0$\pm$0.0
	LTLDoG-S	100.0$\pm$0.0	80.0$\pm$8.1
Temporal behaviors	Diffuser	0.0$\pm$0.0	0.0$\pm$0.0
	LTLDoG-S	96.5$\pm$1.5	66.7$\pm$1.8

(a) Planning.

(b) Rollout C1.

(c) Rollout C2.

Figure 10: Example of failure cases from LTLDoG-S.

TABLE IX: Comparison of Controllers¹ in Avoidance Tasks in Maze2d

Method\Perf.	Success rate ($\%$) $\uparrow$			Reward (UnCon) $\uparrow$
	Planning	Rollout		C1	C2
		C1	C2
Diffuser	9.5$\pm$3.1	11.0$\pm$1.7	15.0$\pm$2.6	142.2$\pm$5.1	92.1$\pm$3.0
SafeDiffuser	99.4$\pm$0.9	12.3$\pm$3.5	14.2$\pm$2.8	135.8$\pm$5.2	131.9$\pm$5.4
LTLDoG-S	99.0$\pm$0.8	73.0$\pm$3.0	90.6$\pm$0.6	97.3$\pm$2.8	36.2$\pm$11.7
LTLDoG-R	98.8$\pm$0.8	92.0$\pm$1.4	99.6$\pm$0.6	127.1$\pm$5.1	76.4$\pm$13.4

• Planned trajectories are rolled out using two different low level control strategies, denoted as C1 and C2.

Fig. 11–Fig. 13 delineate two more examples of the comparisons between our method LTLDoG against the baseline Diffusion Policy on the obstacle avoidance and temporal behavior tasks. These results demonstrate our method’s capabilities in guiding diffusion models to follow not only purely spatial but also spatial-temporal constraints.

(a) Regions.

(b) Diffusion Policy.

(c) Ours (LTLDoG-S).

(d) Ours (LTLDoG-R).

Figure 11: PushT task. Diffusion policy generates next locations of the mover to push the T block by following these locations. Atomic propositions for LTLs are defined by constraints over actions.

(a) LTLDoG-S

(b) LTLDoG-R

Figure 12: Snapshots of two successful rollouts of the two variants of our method accomplishing the same task in Fig. 11. The mover (circle filled in blue) should avoid region $p1$ and $p3$ at all time, as designated in the $\textnormal{LTL}_{f}$ “$\Box(\neg p_{1}\wedge\neg p_{3})$.” Fig. 12(a) and Fig. 12(b) showcase that LTLDoG-S and LTLDoG-R exploit different modalities to complete the original manipulation task while respecting the constraint specified by the $\textnormal{LTL}_{f}$.

(a) Diffusion Policy (baseline)

(b) LTLDoG-S (ours)

(c) LTLDoG-R (ours)

Figure 13: Snapshots of comparisons between the baseline and LTLDoG given an $\textnormal{LTL}_{f}$ with intricate temporal constraints. The $\textnormal{LTL}_{f}$ “$\neg p_{1}\operatorname{\mathsf{U}}{}(p_{2}\wedge(\neg p_{0}\operatorname{\mathsf{U}}{}p_{3}))$” demands the mover never enter $p1$ until (or, unless by the time of entering) the two conditions are both met: 1) it has already been in $p2$, and 2) $p0$ is never reached until $p3$ is visited. Clearly, the baseline breaches this complex constraint when proceeding with the original manipulation task, shown from the third to the fifth snapshot (the mover enters $p0$ without reaching $p3$ beforehand and then gets into $p0$). In contrast, our method, both LTLDoG-S and LTLDoG-R, manage to satisfy the constraint and complete the manipulation task by two different ways (the common ground is to first visit $p3$ and $p2$ during or on the way to maneuvers).

We also show in Fig. 14 more snapshots of a quadruped robot running in the simulated Lab environment with different policies (baseline against ours). The overall satisfaction rate of all raw generated trajectories in different tasks and environments are summarized in Table VIII. The overall performance is $85.8\pm 14.0\%$ (c.f. baseline $2.9\pm 5.4\%$) as reported in the main paper.

(a) Frames of trajectories from Diffuser in simulation.

(b) Frames of trajectories from LTLDoG-S in simulation.

(c) Frames of trajectories from LTLDoG-R in simulation.

Figure 14: Results of safe planning and execution in simulation and indoor room on a quadruped robot.

-D Failure case study

Fig. 10(a) shows an infeasible trajectory generated from LTLDoG-S. Our hypothesis is that posterior sampling in LTLDoG-S calculate gradients based on Euclidean distance on the trajectory space due to the LTL evaluation, which may push the trajectory away from the data manifold. On the other hand, the neural network in regressor guidance is trained on noised intermediate samples, which are linked to the data manifold. Gradients ascent on noise space will likely push the noised example to a place where both the satisfaction value and likelihood are high. Visualizations of the difference between two controllers as shown in Fig. 10(b) Fig. 10(c) suggest that selecting next locations sequentially without distance checking is a less safe option.

Low-Level Control Policy The success rate during rollout also depends on the specific controller used to visit each location of the generated trajectory. TABLE IX shows the comparative results between the two low-level control policies. A more conservative controller (denoted as C2) that visits every location exactly with distance checking can enjoy higher success rate than greedily move (C1, originally used in [1]).