We will first find three position vectors from points A to B, A to C and A to O. To do so, we will first write the locations of points A, B, C, and O in vector Cartesian form.
From the diagram, the locations of the points are:
A:(0i+4j+3k) m
B:(1.5i+0i+6k) m
C:(-2i+0i+4k) m
O:(0i+0j+0k) m
Let us now find the position vectors.
r_{AB}\,=\,\left\{(1.5-0)i+(0-4)j+(6-3)k\right\}
r_{AB}\,=\,\left\{1.5i-4j+3k\right\} m
r_{AC}\,=\,\left\{(-2-0)i+(0-4)j+(4-3)k\right\}
r_{AC}\,=\,\left\{-2i-4j+1k\right\} m
r_{AO}\,=\,\left\{(0-0)i+(0-4)j+(0-3)k\right\}
r_{AO}\,=\,\left\{0i-4j-3k\right\} m
A position vector, denoted \mathbf{r} is a vector beginning from one point and extending to another point. It is calculated by subtracting the corresponding vector coordinates of one point from the other. If the coordinates of point A was (x_A,y_A,z_A) and the coordinates of point B was(x_B,y_B,z_B), then r_{AB}\,=\,(x_B-x_A)i+(y_B-y_A)j+(z_B-z_A)k
We will now find the magnitude of each position vector.
magnitude of r_{AB}\,=\,\sqrt{(1.5)^2+(-4)^2+(3)^2}\,=\,5.22 m
magnitude of r_{AC}\,=\,\sqrt{(-2)^2+(-4)^2+(1)^2}\,=\,4.58 m
magnitude of r_{AO}\,=\,\sqrt{(0)^2+(-4)^2+(-3)^2}\,=\,5 m
The magnitude is equal to the square root of the sum of the squares of the vector. If the position vector was r\,=\,ai+bj+ck, then the magnitude would be, r_{magnitude}\,=\,\sqrt{(a^2)+(b^2)+(c^2)}. In the simplest sense, you take each term of a vector, square it, add it together, and then take the square root of that value.
Next, we will find the dot product between r_{AB} and r_{AO}.
The dot product is the product of the magnitudes of two vectors and the cosine angle \theta between them. For example, if we had two vectors, \mathbf{A} and \mathbf{B}, the dot product of the two is: A\cdot B\,=\,AB\cos \theta. In cartesian vector form, A\cdot B\,=\,A_xB_x+A_yB_y+A_zB_z. The angle formed between two vectors can be found by: \theta\,=\,\cos^{-1}\left(\dfrac{A_xB_x+A_yB_y+A_zB_z}{(\text{magnitude of}\,A)(\text{magnitude of}\,B)}\right)
Now, we will find the dot product between r_{AC} and r_{AO}.