A very recent and similar question was closed asks what "for good" means. While general reference can answer the question, I became curious as to the etymology of the idiom. Googling around got me nowhere, which is unusual. The closest I found was a thread in wordorigins.org:
