State news agency Bernama reported that the four Malaysians were taken away Friday evening by eight armed militants.
The four abducted men, aged between 21 and 34, were among nine crew members of a ship which was heading back to Malaysia from the Philippines, Sabah police chief Abdul Rashid Harun was quoted as saying.
Five others on the ship, who were Myanmar and Indonesian nationals, were released by the armed group, he said.
Meanwhile, Malaysian national police chief Khalid Abu Bakar said the police were still investigating if the abduction involved the Abu Sayyaf group based in southern Philippines.
Earlier this week, authorities said 10 Indonesian crewmen were kidnapped by the terrorist group Abu Sayyaf off Tawi-tawi in southern Philippines.
The 400-strong Abu Sayyaf, founded in the early 1990s by Islamic extremists, is a violent terrorist group operating in southern Philippines. The group is notorious for a series of kidnappings, bombings and beheadings over the decades, both in the Phillipines and across the border in Malaysia. Enditem