Classic Hallucinations: A lone developer spent a weekend trying to run the Llama 2 massive language mannequin on previous, DOS-based machines. Due to the available open-source code, the challenge in the end succeeded. Nonetheless, adapting Llama 2 to the archaic DOS surroundings was no straightforward feat.
Yeo Kheng Meng, a programmer beforehand identified for making a DOS consumer for ChatGPT, has just lately launched into a brand new AI-related challenge centered on the prompt-based computing surroundings of the previous. His newest endeavor is a completely DOS-based massive language mannequin (LLM) that performs inference duties offline.
The FreeDOS Mission notes that Meng developed the DOS LLM consumer utilizing Meta’s Llama 2 mannequin, launched in 2023. The programmer examined the software program on techniques with each older and newer parts, offering a placing side-by-side efficiency comparability between a Thinkpad T42 (2004) and a Toshiba Satellite tv for pc 315CDT (1996). Unsurprisingly, working the LLM on the older system took ages.
He developed an all-inclusive DOS LLM with the assistance of llama2.c, an open-source challenge designed to rapidly port Llama 2 fashions to varied techniques and {hardware} platforms. Meng defined that regardless of llama2.c being written for portability, it nonetheless requires some coding changes to deal with the challenges of classic computing environments.
The programmer needed to choose the best DOS extender to entry bigger RAM swimming pools past the standard 640KB standard reminiscence. He additionally needed to modify the OWC compiler as a result of llama2.c requires a C compiler with comparatively trendy options. After adjusting the compilation course of, Meng created a completely working llama2 binary that ran on his DOS techniques.
The programmer benchmarked a number of techniques working MS-DOS 6.22, from a classic 486 DX2 66MHz PC to a contemporary Ryzen-based desktop. As anticipated, newer techniques delivered sooner inference speeds. Surprisingly, a ThinkPad T42 with a 1.7GHz Pentium M outperformed a ThinkPad X13G1 launched 16 years later.
Meng described the method of testing, documenting, and porting llama2.c to DOS as a weekend challenge. He thanked Andrej Karpathy, who open-sourced llama2.c, for making it potential. With out Karpathy’s and Meng’s work, classic techniques might need prevented the present wave of hallucination-prone AI.
