1. Research Update: Optimal 3TI MPRAGE T1 Mapping and Differentiation of Bloch Simulations
Aug 4, 2014
Jason Su

2. Cramér-Rao Lower Bound
A key theorem in estimation theory and statistics.
The Cramér-Rao lower bound (CRLB) provides a lower bound on the variance of any estimator in the presence of noise. It can:
Analyze the effectiveness of parameter mapping methods and protocols
Optimize their design by finding the protocol that minimizes the CRLB and provides the most precision per unit scan time
However, it has seen limited use in MRI due to the difficulty of analytically deriving the bound for complex signal behavior
Kingsley. Concepts Magn. Reson. 1999;11:243–276.

3. Theory: A common formulation
A desirable property, the mean of many samples converges to the true parameter value
Unbiased estimator
For magnitude images with adequate SNR this holds true
Normally distributed noise
Separate images in a scan protocol have independent noise
Measurement noise is uncorrelated
𝐽 𝑖,𝑗 = 𝛿 𝑔 𝑖 𝑝𝑟𝑜𝑡𝑜𝑐𝑜𝑙 Θ 𝛿 Θ 𝑖
𝐹= 𝐽 𝑇 Σ 𝑛𝑜𝑖𝑠𝑒 −1 𝐽
Σ Θ ≥ 𝐹 −1 𝜎 Θ 𝑖 ≥ Σ Θ ,𝑖𝑖
These assumptions are fairly mild and reasonable for MR in good SNR regime
Note that these are by no means necessary, CRLB can analyze biased and non-normal cases as well. Though biased is harder. Often requires monte carlo simulation.
More specifically, the variance in your T1 estimate is at best the square root of the corresponding entry on the diagonal of the covariance of

4. Theory: Fisher Information Matrix
𝐽 𝑖,𝑗 Θ = 𝛿 𝑔 𝑖 𝑝𝑟𝑜𝑡𝑜𝑐𝑜𝑙 Θ 𝛿 Θ 𝑖
𝐹= 𝐽 𝑇 Σ 𝑛𝑜𝑖𝑠𝑒 −1 𝐽
Definition
gi, the signal equation for the ith image
Θ, the set of parameters, e.g. T1 and M0
J, the Jacobian of the scan protocol
Derivatives of the acquired signal equations for each tissue parameter input
Σ, the noise covariance between the images
This is diagonal with uncorrelated images
Interpretation
J captures the sensitivity of the signal equation to changes in a parameter
F is the Fisher matrix representing the information captured by a protocol
Its “invertibility” or conditioning is how separable parameters are from each other: the specificity of the measurement
This is can be confounded by nuisance parameters which have little diagnostic value but must be estimated, such as M0
Essentially the heart of the CRLB
A given tissue, theta, which means anything that affects your signal equation, T1s, T2s, diffusion, MT, etc.
You can think of it as the projection of the parameters of interest on the space orthogonal to the space spanned by the nuisance variables.
For example, T1 as a nuisance variable in B1 mapping.

5. Theory: Cramér-Rao Lower Bound
Σ Θ ≥ 𝐹 −1 𝜎 Θ 𝑖 ≥ Σ Θ ,𝑖𝑖
The variance of an estimate of a parameter, Θi, is no better than the corresponding ith diagonal entry on the inverse of the Fisher information matrix

6. The Challenge: Computing the Jacobian
Classical computer-based differentiation methods are problematic to apply to this task
Has previously limited the application of CRLB
Difficult by hand, inefficient by computer program
Slow with partial derivatives of multi-input/output functions
Analytic or symbolic differentiation
Prone to round-off errors
Numeric differentiation
An algorithm that solves all these problems
Calculation times comparable to first difference numeric
But 108 times more accurate
Automatic differentiation

7. Optimal Experimental Design
How do we design an experiment to give us the highest SNR measurement of a quantity?
If you only had one shot to examine something for the next 20 years, how would you collect the best data possible?

8. Optimal Experimental Design
The effect of noise on measurements of a nonlinear function is hard to estimate
Often analyzed with computationally expensive Monte Carlo simulations
Impractical for evaluating many scenarios
Unsuitable for the problem of finding the most efficient set of measurement points
We can relax the problem: use the Cramér-Rao Lower Bound instead to predict our experimental precision
This is much more tractable. We just need some derivatives.
This formalism has been used in many scientific fields including dark energy experiments in cosmology.
𝐽 𝑖,𝑗 = 𝛿 𝑔 𝑖 𝑝𝑟𝑜𝑡𝑜𝑐𝑜𝑙 Θ 𝛿 Θ 𝑖
𝐹= 𝐽 𝑇 Σ 𝑛𝑜𝑖𝑠𝑒 −1 𝐽
Σ Θ ≥ 𝐹 −1 𝜎 Θ 𝑖 ≥ Σ Θ ,𝑖𝑖
Atkinson and Donev. Optimum experimental designs. Oxford: Oxford University Press; 1992

9. The Framework
We created a framework that marries the CRLB with automatic differentiation
Enables the calculation of the unbiased CRLB for arbitrary signal equations with no more effort than implementing the equation under study
Including sequences that can only be described with Bloch simulations
Python was used with PyAutoDiff and SciPy for general function optimization
This enables the analysis and optimization of arbitrary parameter mapping experiments
Including highly exotic techniques like MR Fingerprinting
Or one can design new methods by evaluating the information provided by a new pulse sequence
Git it:

10. A Typical Setup
Implement a function that computes the signal(s) for your pulse sequence(s)
Sometimes as simple as a one line signal equation for magnitude data, but it can also be more complex
Create a cost-function
We provide a one line wrapper for your signal function to compute its derivatives
Use our functions to find the CRLB of the scan protocol with these derivatives
Weight the variance of the estimated parameters according to your interest

11. A Typical Setup: Optimization
Set up constraints on your solution space
We provide utilities to accomplish this easily using the actual names of your function inputs
Feed the cost function into a general solver
Find the optimal choice of scan parameters, number of images, and fraction of time to spend on each image series
Profit!

12. Application: 3TI MPRAGE
Liu et al. performed a brute force optimization of the experiment using a noise function
Not sure if noise function is exact
Sampled on a grid and took the minimum
From this he drew some heuristics about what optimum values of TI and TS should be
I reproduce this analysis using the experimental design framework
With greater numerical accuracy (parameters limited by sampling grid)
And extend it to more complex scenarios

13. 3TI MPRAGE: Setup
MPRAGE signal equation implemented in 3 different formulations from literature to ensure correctness
For unbiased CRLB analysis, I think we should be using the raw signal equations, before any image combination
Fisher information/CRLB describes the sensitivity of the actual images collected
Representing the intrinsic lower bound on the pulse sequence itself to changes in T1, instead of a particular recon. implementation
In 3TI MPRAGE, various effects are removed or modeled out in the fitting process
These include: M0 and more perhaps subtly, B1+ or κ
In such analyses, there are two types of parameters, those that we assume a value for and those that we estimate/account for

14. 3TI MPRAGE: Setup Find TIs, TS, <acquisition time fraction>
Estimate T1, M0, κ
Cost: T1/σT1 * sqrt(TS)
The coefficient of variation of T1, normalized by acquisition time
i.e. precision efficiency
Fixed parameters – based on Liu’s marmosat parameters
N = 64 readouts, centric-encoding
α = 9deg., TR = 8.45
Constraints – implemented as plugin transforms to the input variables
train_length = (N-1)TR =
TS ≥ 2*train_length
TI ≤ TS – train_length
TI ≥ 10ms (or higher for multi T1)
Solver
Brute force search number of TIs/images from 2-5
Regularization = 1e-16, checked against neighboring values 0, 1e-14, 1e-12 for consistency

15. Single T1=1300ms
Variable Time Fraction
Equal Time Fraction

16. Single T1=1300ms
Variable Time Fraction
Equal Time Fraction

17. Comments
As Liu observed, the first and last TIs should be as close to the bounds as they can get
The middle TI should be ~T1
Liu stated that TS should be 4.2*T1
This shows about 4.4
Let’s be a bit more rigorous about these conclusions
Solve the single T1 design problem over a range

18. Optimal Parameters as T1 Changess

19. Cost Gap

20. Multiple T1
To optimize over a range of T1s, we sample the range and evaluate the CRLB of a protocol for each
The cost is then a single value summary of these precisions
Common to either minimize the mean or maximum (i.e. worst case) CoV
T1= ms (20 pts)

21. Multi T1=
Mean
Max

22. Multi T1=
Mean
Max

23. Multi T1= , equal time
Mean
Max

24. Multi T1= , equal time
Mean
Max

25. Multi T1=1000-2000, free FA, equal time
Mean
Max

26. Multi T1=1000-2000, free FA, equal time
Mean
Max

27. Next Steps
Multi κ
Multiple α recon?
ARLO?

28. Bloch Simulation
One of the distinct advantages of automatic differentiation is that it can handle complex programs
Bloch simulation and extended phase graph analysis are ways to analyze the MR experiment using computer programs
More complicated mapping methods, like MR Fingerprinting, rely on simulation to describe their
This provides a way to extend the experimental design framework to more exotic pulse sequences

29. Libraries I’ve been using pyautodiff for AD
It provided the most seamless conversion of functions to derivatives, with 0 extra code asked of the user
However, it is slow. Somewhat surprising because it uses theano which does a JIT compile to C.
It is even slower for programs with loops
So I went shopping for different AD packages
theano (what I’ve been using)
ad (pure python)
algopy (inspired by ADOL-C?)
CasADi (python frontend to C library)

30. Simple Speed Test
Bloch simulation is essentially a sequence of matrix multiplies and additions on an input time series
Simple vector accumulation test
Other ones I want to try:
CppAD, pyadolc
Package
F (1k x 1)
dF (1k x 1k)
Python
0.848ms
theano
4.34ms
8640ms
algopy
271ms ad
DNF
CasADi
0.0573ms
8.64ms

31. Implementation
Given that there are still other AD packages out there that may be better
Bloch simulation implemented to be modular so can plugin in whatever AD library
Assuming it has the same general structure:
Instantiate symbolic tracer variables
Use the specific math functions from the library
With this I have theano and CasADi versions working

32. Bloch Simulation Speed
For a 1000 length input Bloch simulation:
Hargreaves’s MEX = 0.13 ms
CasADi = 1ms
theano = 94ms
For a 1000x1000 Jacobian of the simulation:
Central difference = 0.1*2000 = 260ms
Half for forward difference
CasADi = 115ms (with no loss of accuracy!)
theano = 2m54s

33. Should I migrate everything to CasADi?
Or away from theano
Ease of use?
Been treating bloch as a separate module so don’t implementation can be as complex as I can take