Overview

We tested five large language models (LLMs) to evaluate their ability to independently convert a website using Client-Side Rendering (CSR) with React into Server-Side Rendering (SSR) with Next.js. The models tested were:

- ChatGPT

- DeepSeek

- Gemini

- Devin

- Llama

Key Findings

• None of the models tested were able to achieve the given task independently.

• If an experienced pool of programmers is available, it is more efficient to migrate on their own. Currently, none of the LLMs we tested can deal with several critical aspects, so an experienced developer is needed no matter what. In that case, a developer can be more efficient just doing it directly by themselves, without LLM instructions which oftentimes can be unreliable.

• Inexperienced users might follow LLM instructions for simple tasks, but long-term maintainability and debugging would still require expertise.

• We believe that the current versions of the LLMs we tested would be helpful for quicker onboarding of an inexperienced/junior developer on new programming languages.

Detailed Model Evaluations

ChatGPT 4o

• ChatGPT 4o can read code from uploaded a ZIP files and convert it, allowing you to download the results directly. However, the code inside remains unchanged—it’s simply copied and pasted without any modifications.

• Also utilized o3-mini-high version for further check, file uploads are not supported; requires manual copy-pasting. The conversion was better but still not complete and failed on full project migration.

Gemini Pro

Limitations

• Gemini mainly focuses on language processing. Its ability to understand and convert code, especially for tasks involving multiple files and complex dependencies like project conversion, may not be fully competent.

• Gemini may not currently support directly processing ZIP files or JSX format files, which limits its ability to operate at the project level.

Test results

• A test done by pasting the official website's App.JSX and verified that Gemini can perform code conversion, but only for single file conversion, and it requires manual intervention to confirm details.

• Gemini cannot directly complete the conversion of a Next.js project and still requires the intervention of a front-end engineer.

Deepseek

• Compared to Chatgpt, DeepSeek has a better understanding of how frameworks like React.js and Next.js work and it can give well-detailed, accurate examples that are clear to understand, even if the prompt is not polished. 

• DeepSeek has a DeepThink(R1) feature specifically for tasks that require complex logical thinking, mathematical problem-solving, and detailed explanations to a conclusion. The user can see which part of the prompt DeepSeek is uncertain of to improve the prompt for a better outcome.

Limitations

• Currently, it is not scaled enough for the demand. At times, it just returns to “The server is busy. Please try again later”.

• Unable to accept zip files.

• Sometimes it can be misleading about what it is capable of. For example, it claims to be capable of analyzing a repository via its GitHub link but when a repo link is provided claims it is not capable.

Personal opinion by a developer: "Deepseek performed better than Chatgpt when it was asked about migrating the React app to the Next app. Even though it is not powerful enough to access the repository and manipulate it, it gives a clear and concise guide on how to go about it. Also, it gives out more relevant pieces of advice and tips that are not specifically asked in the prompt. If its scaling is improved, it is preferable over Chatgpt, since it seems to have a more critical thinking process."

Devin

• It was tested by prompting it for bulk file migration from React.js to Next.js, and Devin was not able to access any ZIP files. The only way to bypass that is manual copypasting of code snippets into chat threads which is not ideal.

• Devin is able to clone a repository and make changes to the code inside its designated IDE, however, to test it, you are required to buy a subscription that costs 500 USD/month. Further research was done on Devin by reading articles and watching reviews from programmers who had experience in using a paid version of Devin, and the results were not to the level of satisfaction.

• Processing is too slow, not able to see real-time changes in your local IDE (there is a mandatory need to go to Devin's remote IDE to look for results for confirmation), results are not precise, and it doesn't reach the hype.

Llama

• By using Llama, transfer attempts were made by, first uploading multiple CSR files to convert into SSR, but the results were unsatisfactory. The model treated the task as a sample project and returned only a minimal SSR example rather than a true file-by-file conversion.

• To be able to move forward, two main optimization directions were identified:

1. Improve prompts by providing the model with a single-file example plus expected outcomes, and potentially increasing model parameters (though that would require more GPU RAM).
2. Hardware considerations: Converting thousands of files is time-consuming on the local device, further resulting in the device's significant overheating. Additional or more powerful hardware (more RAM/GPU resources) might be required to handle the workload efficiently.

Final Verdict

• Current LLMs cannot replace experienced developers for complex tasks like CSR-to-SSR migration.

• They are best used as assistive tools for learning, debugging, and small-scale automation.

📢 What are your thoughts? Have you tested other LLMs for development tasks? Let us know!

Should We Test Additional Models?

Initial tests suggest that Grok has potential, but a full field report has not yet been conducted. We welcome suggestions for other models that may be worth testing.

4i Tech