Core curriculum - Mathematics: Linear algebra VI

 

A13. Eigenvectors and Eigenvalues

Let's end this section on linear algebra with a brief exploration of eigenvectors and their eigenvalues. An eigenvector is simply one which is unchanged by a linear transformation except to be scaled by some constant. The constant factor (scalar) by which the eigenvector is scaled is called its eigenvalue. If the eigenvalue is negative then the direction of the vector is reversed as well as scaled.

 Curious about the terminology? Eigen means "proper" or "characteristic" in German. So if you're struggling to understand or remember what eigenvectors are all about, perhaps it helps to rename them "characteristic vectors" instead.

Here's a nice introduction to the concepts. Pay close attention to the symmetry arguments. It turns out eigenvectors represent things like axes of rotational symmetry and the like:

 

 

And with some of the insights under your belt, here's a tutorial on the mechanics of finding eigenvalues and eigenvectors:

 

________________

Core curriculum - Mathematics: Linear algebra V

 

With some understanding of basic matrix manipulations, we're ready to begin using matrices to solve systems of linear equations. In this post, you'll learn a few standard tools for solving small systems - system defined by a small number of equations - by hand. Naturally, larger systems as found in fMRI will use computers to solve the equations, but you should understand what's going on when you push the buttons.

A11. Elementary row operations and elimination

 

This is just your standard algebraic manipulation to solve multiple simultaneous equations, e.g. dividing both sides of an equation by some constant to be able to simplify, but where the equations are represented as matrices:

 

A12. Cramer's Rule for solving small linear systems

According to Wikipedia:

In linear algebra, Cramer's rule is an explicit formula for the solution of a system of linear equations with as many equations as unknowns, valid whenever the system has a unique solution. It expresses the solution in terms of the determinants of the (square) coefficient matrix and of matrices obtained from it by replacing one column by the column vector of right-sides of the equations.

 

________________

Core curriculum - Mathematics: Linear algebra IV

 

Before getting back to the lectures from 3Blue1Brown, try this part review, part preview:

Now let's get back into the meaning with a little more detail.

 

A9. The dot (or scalar) product 

The dot product is a way to estimate how much two vectors interact in a common dimension. If the vectors are orthogonal to each other, they don't interact in a common dimension so their dot product is zero. This is like asking how much north-south movement is involved in an east-west heading: none. But if two vectors are perfectly parallel then this is equivalent to the two vectors lying on the number line and we can use our standard (scalar) multiplication rules. In between, we use a little trigonometry to determine their (dot) product.

 

Still lacking an intuition? This excellent summary from Better Explained (slogan: "Learn Right, Not Rote") should do the trick.

A10. The cross (or vector) product

Both the dot and cross products affect dimensionality. With the dot product, we find how much two vectors interact in one dimension. The cross product of two vectors is perpendicular to them both, telling us how much rotation arises in a third dimension.

A useful real world example use of the cross product is to compute the torque vector. Torque is the rotating force generated by pulling or pushing on a lever, such as a wrench or a bicycle crank. The lever moves in one plane but produces a rotation orthogonal to that plane. 

 

 

Torque is also fundamental to the origins of the MRI signal. We will encounter it later in the physics section. Can you take a guess how torque might be relevant to the MRI signal? Hint: it has to do with the interaction of a nuclear magnet (the protons in H atoms) with an applied magnetic field.

This article from Cuemath covers the rules for computing dot and cross products. And here are a couple of useful visualizations:

 

 

________________

 

 

Core curriculum - Mathematics: Linear algebra III

 

Now we start to think about transformations between dimensions, e.g. taking a 2D vector into a 3D space. Non-square matrices come up frequently in engineering and research applications, including fMRI analysis, so you'll want a good understanding of their meaning. 

 

 A8. Non-square matrices

Let's look at a simple physical interpretation of changing the number of dimensions.

We previously saw how to invert a square matrix. But how do we invert a non-square matrix?

 

________________

Core curriculum - Mathematics: Linear algebra II

Continuing the series on linear algebra using the lectures from 3Blue1Brown, we are getting into some of the operations that will become mainstays of fMRI processing later on. It's entirely possible to do the processing steps in rote fashion as an fMRI practitioner, but understanding the foundations should help you recognize the limits of different approaches.

A4. Matrix multiplication as composition

In this video we see how to treat more than one transformation on a space, and how the order of transformations is important.

 

Q: While brains come in all shapes and sizes, we often seek to interpret neuroimaging results in some sort of "average brain" space, or template. We need to account for the variable position and size of anatomical structures. However, we also have the variability of where that brain was located in the scanner, e.g. because of different amounts and types of padding, operator error, and so on. When do you think it makes the most sense to correct for translations and rotations in the scanner: before or after trying to put individual brain anatomy into an "average brain" space? Or does it not matter?

 A5. Three-dimensional linear transformations

 Now we're going to move on from 2D to 3D spaces. Same basic rationale, just more numbers to track!

 

A6. The determinant 

 

A7. Inverse matrices, column space and null space

 

 

Perhaps it's not fully clear why we might need the inverse matrix. It turns out to be the way to achieve the equivalent of division using matrices. To galvanize this insight, let's look at the concept of an inverse matrix for solving an equation without division. Leaving aside the slightly goofy intro, it's a useful tutorial on the mechanics of determining an inverse matrix. 

________________

 

Core corriculum - Mathematics: Linear algebra I

 

What is linear algebra? To get us going, I'm going to use the excellent lecture series by 3Blue1Brown and do my best to add some MRI-related questions after each video. Hopefully the connections won't be too cryptic. Don't worry if you can't answer my questions. It's more important that you understand the lectures. No doubt you'll find other material on YouTube and web pages to clarify things.

Let's start with a couple of definitions. While you'll find many examples online, for our purposes we can assume that a linear system is one where the size of the output or outputs scales in proportion to the input or inputs. The take-home pay of a worker paid an hourly rate is linear. They might receive their base amount, say 40 hours per week, plus some amount of overtime at twice their hourly rate. The total is still the linear combination of the base plus overtime amounts.

Non-linear systems don't have this simple proportionality. Gravity is the classic physics example. The strength of the interaction between two massive objects changes as the reciprocal of the squared distance (r^2) between them, that is, as 1/r^2. Finding yourself dangling ten meters in the air above the earth is very different from finding yourself ten more meters away from the earth at a height of 1000 km. In the first case you are about a second away from impacting the ground. In the second case you are in orbit and your more immediate health concerns are lack of oxygen and your temperature.

And what about the term algebra? It's just fancy speak for using symbols to represent the relationships between things that vary. We're going to be interested in changes at different positions in space - points in an image - and so we shall eventually use matrices to perform linear algebra. But we have to build up to a matrix from its skinnier cousin, the vector.

A1. Vectors: Essence of linear algebra

 

Q: We will use both a physicist's and a computer scientist's view of vectors at different points in the fMRI process. Given what you know today, can you guess where these different viewpoints might come up? Hint: fMRI is based on MRI, which is a physical measurement technique, while fMRI is typically the analysis of a time series of a certain type of dynamic MRI scans.

 

A2. Linear combinations, span, and basis vectors

 

 

We might as well look at changes of basis at this point:

 

 

Q: Changes of basis are quite common in MRI. Even the way we usually label image axes involves a change of basis. The magnet bore direction is labeled the z-axis, while left-to-right is the x-axis and up-down is the y-axis. We refer to this assignment as the lab (or magnet) frame of reference. Now consider an axial MR image of a person's brain. An axial slice lies in the x-y plane in the magnet basis (or lab frame if you prefer). Yet we don't generally label the image with (x,y) dimensions. Instead we use (L-R, A-P) where L-R is left-to-right and A-P means anterior-to-posterior. This is an anatomical basis. How might an anatomical basis be more useful than using a magnet basis in MRI?

A3. Linear transformations and matrices:

 

Q: We usually label images using a basis (or reference frame) related to the subject's anatomy, i.e. with the (orthogonal) axes labeled head-to-foot (HF), left-to-right (LR) and anterior-posterior (AP). This means if a subject's head isn't perfectly straight in the magnet - let's say, the head is rotated 20 degrees to the left - the brain still appears straight in the 2D image. But here's the thing. The MRI hardware is controlled using the (x,y,z) "lab" reference frame. The anatomical and lab bases can be related to each other through a rotation matrix. Can you write down what a rotation matrix might look like that relates the subject's anatomical reference frame to the scanner's lab (x,y,z) reference frame?

________________

Physics for understanding fMRI artifacts: Part Eleven

Resolution and the field-of-view as seen in k-space

Understanding how distances in k-space manifest as distances in image space is quite straightforward. All you really need to remember is that the relationships are reciprocal. The discrete steps in k-space define the image field-of-view (FOV), whereas the maximum extents of k-space define the image resolution. In other words, small in k-space determines big in image space, and vice versa. In this post we will look first at the implications of the reciprocal relationship as it affects image appearance. Then we'll look at the simple mathematical relationships between lengths in k-space and their reciprocal lengths in image space.


Spatial frequencies in k-space: what lives where?

I mentioned in the previous post that there's no direct correspondence between any single point in k-space and any single point in real space. Instead, in k-space the spatial properties of the object are "turned inside out and sorted according to type" (kinda) in a symmetric and predictable fashion that leads to some intuitive relationships between particular regions of k-space and certain features of the image.

Here is what happens if you have just the inner (left column) or just the outer (right column) portions of k-space, compared to the full k-space matrix arising from 2D FT of a digital photograph (central column):

An illustration of the effect of nulling different regions of k-space from a full k-space matrix, applied to a digital picture of a Hawker Hurricane aircraft. The full k-space matrix and corresponding image are shown in the central column.

Inner k-space only:

The inner portion of k-space (top-left) possesses most of the signal but little detail, leading to a bright but blurry image (bottom-left). (See Note 1.) Most features remain readily apparent in the blurry image, however, because most contrast is preserved; image contrast is due primarily to signal intensity differences, not edges. If this weren't true we would always go for the highest signal-to-noise MRIs we could get, when in practice what we want is the highest contrast-to-noise images we can get! Imagine an MRI that had a million-to-one SNR but no contrast. How would you tell where the gray matter ends and the white matter begins? Without contrast no amount of signal or spatial resolution would help. So much for SNR alone!

Outer k-space only:

If we instead remove the central portion of k-space (top-right) then we remove most of the signal and the signal-based contrast to leave only the fine detail of the image (bottom-right). Strangely, though, it's still possible for us to make out the main image features because our brains are able to interpret entire objects from just edges. In actuality, however, there is very little contrast between the dark fuselage of the Hurricane, the dark shadow underneath it and the dark sky. Our brain infers contrast because we know what we should be seeing! If we were to try doing fMRI, say, on a series of edges-only images we would run into difficulties because we process the time series pixelwise. With a relatively low and homogeneous signal level you can bet good money the statistics would be grim.

Physics for understanding fMRI artifacts: Part Ten

(For the answer to the homework k-space diagram given at the end of Part Nine, see Note 1.)

K-space in two dimensions

As anyone knows who has encountered MRI professionally, whether in research or medicine, there seems to be an endless array of pulse sequences to choose between. The variety can be overwhelming at first. Nor is the situation helped by different vendors using different acronyms - we always use acronyms in MRI! - for what are essentially the same sequence.

It's little wonder, then, that most neophytes' eyes glaze over when it comes to comparing and contrasting any two pulse sequences if the taxonomy appears to be ad hoc. Where on earth to start? But it turns out that most pulse sequences can be categorized fairly easily, and their heritage traced, by separating the part(s) of the sequence that is responsible for spatial encoding, from the part(s) of the sequence that will provide the tissue or functional contrast. Occasionally there is overlap within the sequence of these two missions, but even then it's usually straightforward to understand the spatial encoding and interpret its genesis.

A useful pictorial representation of imaging pulse sequences

It turns out that there are only a handful of spatial encoding methods in common use these days, almost all with roots in the late 1970s or early 1980s. While new pulse sequences appear in the literature all the time, when you look at their k-space representations you'll be able to see how each new method has developed from a small number of key ideas from those early years. It's possible to categorize the encoding methods without k-space, but the k-space formalism makes comparisons trivial (in MR terms).

Spatial encoding methods can be separated into families derived from a central idea. For instance, following Lauterbur's original imaging paper in 1973 (which led to the family of projection reconstruction methods), in 1975 Richard Ernst's group came up with a sequence that utilized a 2D Fourier transform to yield the final image. (See Note 2.) It was a remarkable breakthrough and is the grandparent of nearly all medical/biological sequences still in common use today.

Still, even geniuses miss opportunities every now and then. And in 1980 a group at Aberdeen came up with a far more practical implementation of Fourier imaging, using amplitude-modulated gradients in a "constant time" pulse sequence, rather than the fixed amplitude, variable time scheme of Kumar, Welti and Ernst. It is this constant time scheme, which the Aberdeen group termed "spin warp" phase encoding, that provides the basis for most clinical (anatomical) scanning used today. It's also a good scheme to look at when first encountering 2D k-space, so we'll consider it in detail in this post.

The goal revisited

In the first part of the last post (see Part Nine) I used two examples of digital images to illustrate how the information content in a 2D plane of image pixels can be equivalently represented in reciprocal 2D space, or k-space. I mentioned that both the images and the k-space comprised 512x512 points, but later on when I started to draw (one-dimensional) k-space trajectories I did so on a k-space plane that was represented by just a set of axes, not discrete points. In case you think that image space and k-space in MRI are continuous, I'm going to spend a moment considering the digital k-space plane explicitly. (Like real space, k-space can also be continuous rather than digital, but that's not how MRI works.)

Here is a 16x16 plane of k-space points (see Note 3) overlaid on some actual signals to reinforce the point that we're digitizing a continuous process:

Courtesy: Karla Miller, FMRIB, University of Oxford.

The goal is to traverse the entire k-space plane, i.e. to use our gradients to follow a trajectory that crosses every single point (as defined by the white grid itself), acquiring data (with our receiver coil), one point for each grid coordinate, as we go. Once we have traversed the entire 2D plane (and assuming a suitable data acquisition scheme) we will have 16x16 k-space data points and will then be in a position to apply a 2D FT and get a 16x16 image out. (See Note 4.)

Physics for understanding fMRI artifacts: Part Nine

Conjugate variables redefined

In this post I'm going to provide the first part of a recipe for generating 2D images. It's going to be somewhat algorithmic. I may occasionally mention what a particular step implies, but for the most part I'm going to step through a sequence of events, produce a final recipe for you to follow, then go back and explain what some of the parts mean physically. This isn't the traditional approach to learning about k-space; most text books assume that you need to understand what it all means before you get to learn "the rules of the game." As is my wont, I'm coming at it backwards. My hope is that you will then be able to go back to your text books - I'll tell you where to look for subsequent explanations - and cement a decent understanding of the "why" of k-space, not just the "how."


Conjugate variables revisited

In Part Five of this series I introduced the Fourier transform and conjugate variables. The post focused on the most common pair of conjugate variables: frequency and time. If we have the time domain representation and we want to transform it into its frequency domain equivalent, we apply a (one-dimensional) FT, and vice versa.

But there is another pair of conjugate variables that is more useful and intuitive for imaging applications. (In this case your intuition for one of the variables may not develop until the end of this post, or later! Bear with me.) Whether it's maps, MRIs or architectural plans, the axes of an image are best described in terms of length. If we choose the centimeter as our unit of length, then FTing an axis in cm will yield an axis in 1/cm. You happen to have an intuitive notion of time, frequency and space from everyday life. Don't worry about what the reciprocal of real space means, just accept for now that it exists. We call this reciprocal space k-space because another term for 1/cm is the wavenumber, and the wavenumber is given the symbol k.

Representing pictures in reciprocal space

Let's take a random picture, in this case it's a digital photograph of a Hawker Hurricane plane. It's clearly a 2D picture. We have a digital version of it so we can do mathematical operations on it with a computer. If we do a 2D (digital) FT of the picture we get its representation in 2D k-space:

Physics for understanding fMRI artifacts: Part Six

Practical issues arising from the use of the Fourier transform in MRI

Here's the plan for this post. We will complete our look at functions undergoing Fourier transformation; there are some really useful relationships to see and to commit to memory (even when the figures are hand drawn for expediency!). Then we will look at the effects of the FT on real signals. We have two issues to consider: 1) a finite sampling window, and 2) digitization. Off we go!

Fourier pairs

We saw in the last post (Part Five) how a single frequency - a sinusoid - can be represented by a single line - a delta function - in a frequency domain plot. This is an example of a Fourier pair because the relationship holds both ways, i.e. if you take a delta function in the time domain and FT it you get a sinusoid in the frequency domain, and vice versa:

What about other useful Fourier pairs? Here's a useful relationship. An exponential decay Fourier transforms into what's known as a Lorentzian line:

Again, remember that the exponential can be in either the time domain or the frequency domain, although in MRI we generally deal with exponential decays (signals) in the time domain. It's also worth pointing out here that the faster the exponential decay, the broader the Lorentzian line in the other domain. This inverse relationship has a number of practical consequences for fMRI. I'll come back to this point below.

I found an interactive online tool on the National High Magnetic Field Lab's website that allows you to change the rate of decay as well as the frequency of an oscillation in the time domain, and see the resulting Lorentzian line in the frequency domain. Tinker with it here. (It's Java, it takes a couple of seconds to load.)

Next, let's look at what could be arguably the most useful Fourier pair for MRI. A boxcar (or top hat) function Fourier transforms into a sinc function, where sinc(x) = sin(x)/x:

Physics for understanding fMRI artifacts: Part Five

An introduction to the Fourier transform - what it does and how it works


Moving along from bathroom design and complex numbers, it's time to look at one of the most fundamental mathematical relationships underpinning all of modern MRI: the Fourier transform (FT). Accordingly, I would strongly suggest that you make sure you fully understand everything in this post before continuing into future posts.

You may not believe it by looking at us, but take it from me that MR physicists do "mental FTs" every day as we compare pulse sequences, try to understand artifacts and so on. Many of us (me) are not mathematically inclined, either. But being able to switch your mind between domains, as they are called, is a really useful skill if you want to be good at artifact recognition. An MR physicist will see something in one domain and will immediately try to imagine how it would appear in the alternate domain. Thus, if an image contains an artifact - an intense stripe, say - the MR physicist tries to imagine what signal-acquisition domain feature is implied by it, then tries to track it down. Alternatively, when comparing two different pulse sequences - GRAPPA on versus GRAPPA off, perhaps - the MR physicist will project the implications of each into the image domain in order to comprehend the likely practical consequences. You don't have to remember the equations or even understand the maths itself, but it is really useful to grasp the concepts!


Fourier analysis: the art of decomposition

As you have seen in your introductory texts and in previous video lectures, MRI signals are actually time-varying voltages that are induced in receiver coils as a result of magnetization oscillating (precessing) about the polarizing magnetic field. The detection of time-varying signals has several implications for obtaining MR images, the very first of which is choosing a representation for the information content in the signals.

Consider the two waveforms on the left-hand side of the figure below, which have the same amplitude but differ in their frequency:

Courtesy: Karla Miller, FMRIB, University of Oxford.

It is possible in principle - and in practice for simple examples such as these - to take a ruler and measure the amplitude and frequency and then draw a graph representing the time varying signals as a plot of amplitude (along y) against frequency (along x), as shown on the right-hand side of the above figure. Piece of cake, right?

Physics for understanding fMRI artifacts: Part F(f)our

(Wondering why the title has F(four) in it? It's so that Blogger can't trash this post for a third time! I'm giving it a new, unique name. Ha!)

It's finally time to get back to the series of posts on the essential physics concepts that will allow you to interpret and differentiate between acquisition artifacts. There are another five or six posts in this background series, so bear with me. After that we will shift gears and look at "good data," taking some time to assess the normal variations that you can expect to see in time series EPI, and then I promise we'll look at artifacts themselves.


When reality meets imagination

Before we go any further there are a few mathematical properties we need to review. These are actually quite simple relationships that, for the most part, can be explained via a handful of pictures. Like this one....

Maths dude, chillin'.

Complex numbers

The name notwithstanding, complex numbers are quite straightforward to understand from a physical perspective, with a tiny bit of explanation. By the time you've finished reading this post you should have a basic idea of what complex numbers mean and where they come from (they arise quite naturally, as it happens), but for now I am simply going to define some relationships. Hang in there.

We start by defining a so-called imaginary number as any number for which the square is negative. The squares of real numbers - the ones you're used to in everyday life, such as 2, 8.73, -7, pi, and so on - are always positive, whether the number being squared is positive or negative. Thus 2x2 = 4, 8.73x8.73 = 76.2129, -7x-7 = 49 and so on. Squaring a negative number results in a positive number. So how could we possibly get an answer of -4 or -25 out of any square?