Skip to main content

Supercomputer Temp

vim​

Vim Usage

Shell Script Editing​

Shell Script Editing (Part 1)

[Shell Script Editing (Part 2)

Software Installation​

conda+module

Compiling from Source

Configuring Conda Environments​

Current Environments (conda)

LLM----Running Qwen, Qwen1.5 series models

GLM3----Running ChatGLM3 model

Baichuan----Running Baichuan2 series models

Conda Folder Copying​

Directly copy the created virtual environment folder to the supercomputer's Anaconda virtual environment directory.

Supercomputer virtual environment directory location:

/data/home/202331060205/.conda/envs

Conda TXT Environment Migration​

  1. Migrating Environments

Once you are ready to migrate the environment to a machine with NVIDIA GPUs, you have several options:

  • Copy Virtual Environment: You can simply copy the entire virtual environment folder to the target machine. However, this method may encounter compatibility issues due to different operating systems or architectures.

  • Export Dependency List: A more reliable method is to export the dependency list of the current environment, then recreate the virtual environment on the target machine and install the dependencies.

    • Using pip:

      pip freeze > requirements.txt

    • Using conda:

      conda list --export > requirements.txt

  • On the target machine, recreate the virtual environment and install dependencies based on the exported requirements.txt file. Make sure that the NVIDIA drivers, CUDA Toolkit, and cuDNN compatible with the model are installed on the target machine.

Supercomputer User Working Directory​

/data/home/202331060205/perl5/NeverGpDzy/Qwen1.5-7bChat
/data/home/202331060205/DingZhiyu/NeverGpDzy/Baichuan2-13B-Chat

Import Python UTF8 Environment Variable​

export PYTHONIOENCODING=utf8