Noticed this as well, I've got >60k posts :S
It seems that the 'post' field, that allows selection of a post, metabox will query all available options for the field on every request. I think it should query available options only when the actual meta box field is being rendered.
I'm not using metabox frontend functions so I was able to narrow this problem to dashboard only by not registering metaboxes on the frontend requests.