By the book
Correctly; according to the rules.
Which book? Well, the Bible sounds like a good guess. That may be so, as the early meaning of the phrase was 'I swear it to be true', as in the oaths taken in courtrooms. An example of that usage, is recorded in The Times, January 1833:
"Patience in troth! By the book, it's myself is the moral o' patience!"
The present meaning also emerged around the same time. The earliest citation I can find is from the mid-19th century - in Edgar Allan Poe's Murders in Rue Morgue, 1845:
"To have a retentive memory, and to proceed by 'the book', are points commonly regarded as the sum total of good playing."