In the default word processing mode, you can only rearrange section thumbnails, but not individual pages within those sections. The optional Page layout mode allows one to rearrange individual thumbnails as they are single sections.
In word processing mode, you have to place your insertion pointer after the last character on that last page, and then choose Insert menu > Page break. Copy/paste the content of Page 2 into that new page at the end of your document. With the content of Page 2 still selected, hit backspace to remove that page content, and then another backspace to delete that page.