Around 1815, VanWormer moved to the west side of the state of New York to Ellisburgh in Jefferson County, where several of his children had previously moved.

He and his wife spent their remaining years there. She died in 1825, and he died in August 1829. Family history records indicate they both were buried in the Woodville Cemetery in Jefferson County, but there are no readable headstones left there to indicate the actual burial locations.

The story of Jacob VanWormer’s service has been handed down from generation to generation in the VanWormer family. Jacob VanWormer was firmly of the opinion that he fired the first gun at Bennington and killed a British officer. He participated in the battles of Bemus Heights, fought September 19, and October 7, 1777, and the engagement 10 days later which resulted in the surrender of Burgoyne. Much of his service was rendered in his capacity as minuteman, ranger and scout.

In 1778, a year after the adoption of the Constitution of the State of New York, the military forces of the state were reorganized. Jacob VanWormer was mustered as a private in the third (Hoosick) Company, in the Fourteenth Regiment, Albany County Militia, which was commanded in succession by Colonels Knickerbacker, Rensselaer, Gates and VanWess. He later became an ensign and second lieutenant in the company and regiment. His military service continued until the declaration of peace.