If an accretion disc contains weak frozen-in entangled magnetic fields, their dynamical effect may be important inside the last stable orbit because of decompression near the sonic point. Here, I consider the radial and vertical structure of a nearly free-falling flow inside the last stable orbit of a thin disc around a Kerr black hole. The thickness of such a flow is determined primarily by the vertical stress created by radial and azimuthal magnetic fields. The thickness is predicted to oscillate vertically around its equilibrium value, determined by the magnetic field balance with gravity. For thin discs, this thickness is much larger than that of the accretion disc itself. Numerical simulations with HARM2D (High Accuracy Relativistic Magnetohydrodynamics) show that the vertical structure is more complicated. In particular, a magnetically supported disc seems to be unstable to segregation of matter into thinner streams, with the vertical scale determined by thermal pressure or other processes.