1. Introduction

 

 

 

 

 

 

 

 

 

 

 

 

 

PROMPT ENGINEERING

 

(SEC)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table of Contents

 

S.No	Experiment
1.	Environment & Connectivity: Install required packages (e.g., transformers, openai); securely configure the API key; run a simple “Hello, world” prompt to verify model access.
2.

 

 

 

 

 

 

 

 

 

 

 

Introduction

Large Language Models (LLMs):

Online Variant

Online Large Language Models (LLMs) are accessed through cloud-based APIs or web interfaces. Examples include OpenAI API services. These models provide high-quality responses, scalability, and access to state-of-the-art capabilities. However, they generally require internet connectivity, API credentials, and may incur usage costs.

Local Variant

Local LLMs execute directly on a user's computer using frameworks such as Transformers. Models like DistilGPT-2 and FLAN-T5 can be downloaded and run offline. Local execution improves privacy and eliminates API costs, although performance depends on the available hardware resources.

Comparison of Online and Local Variants

 

Feature	Online	Local
Internet	Required	Not required after download
Billing	Usually required	Free
Privacy	Cloud processing	Local processing
Hardware	Minimal	User dependent
Setup	API key configuration	Model download

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Exp:1 Environment & Connectivity

 

AIM: Install required packages (e.g., transformers, openai); securely configure the API key; run a simple “Hello, world” prompt to verify model access.

 

THEORY:  Local environment is capable of running a Large Language Model (LLM) on a personal computer, and to verify that the model loads and generates a response without depending on an external/cloud API.

Cloud-based LLM APIs require continuous internet access, recurring usage cost, and involve sending data to a third-party server, which may not be suitable for sensitive data, offline use, or learning purposes.

ADVANTAGES OF LOCAL LLM:

        

        •         Data privacy – input/output never leaves the local machine.

•         No recurring API cost and works without internet once the model is downloaded.

•         Full control over the model version and ability to fine-tune or experiment freely.

 

PROCEDURE:

Pre-requisites / Tools Used

•         Windows OS with internet connection (for one-time downloads)

•         Microsoft Visual C++ Redistributable (x64)

•         Python 3.12.7

•         Python packages: torch, transformers, accelerate

Installation Steps

Install Microsoft Visual C++ Redistributable

1.      Open the official Microsoft download link in a browser:

            https://aka.ms/vs/17/release/vc_redist.x64.exe

2.      Download the file vc_redist.x64.exe.

3.      Run the installer as Administrator and accept the license terms.

4.      Click Install and wait for completion, then click Close.

5.      Restart the system so the runtime libraries load correctly.

 

        Install Python 3.12.7

6.      Go to the official Python website: https://www.python.org/downloads/release/python-3127/

7.      Download the “Windows installer (64-bit)” for version 3.12.7.

8.      Run the installer and tick the checkbox “Add python.exe to PATH”.

9.      Click custom installation and specify Installation folder and wait for setup to finish.

10.  Verify installation by opening Command Prompt and typing:

           

            python –version

 

            Expected output: Python 3.12.7

 

Create Virtual Environment and Install Packages

 

1.      Create a project folder and a virtual environment to keep packages isolated.

 

       cd D:\localLLM

       python -m venv llm-env

llm-env\Scripts\activate

 

2.      Install the required packages inside the activated environment.

 

       pip install transformers torch accelerate

            transformers is Hugging Face's library for loading models, torch is the backend     that actually runs the math, and accelerate helps manage devices (CPU/GPU)       automatically

 

3.      Verify torch installed correctly and check GPU availability.

 

     python -c "import torch; print(torch.__version__,      torch.cuda.is_available())"

 

This prints the torch version and whether a GPU is detected (True/False). CPU-only is fine for small models, just slower.

 

Program

 

The following Python program loads a small local language model and generates a response to a test prompt, verifying that the local LLM environment works correctly

 

 

from transformers import pipeline

 

# Load a small local model (downloads once, then runs offline)

generator = pipeline("text-generation", model="gpt2")

 

# Generate a response to a test prompt

response = generator(

    "Hello, world! Tell me something interesting:",

    max_new_tokens=30

)

print(response[0]["generated_text"])

 

Execution – Steps Explained

1.      Save the program above as hello_local_model.py inside the project folder.

2.      Ensure the virtual environment (llm-env) is activated before running.

3.      Run the script using the command below; this loads gpt2 and generates text.

 

python hello_local_model.py

 

 

4.      The pipeline() call downloads the model files once and caches them locally for future runs.

5.      The generator object then runs the model on the given prompt entirely on the local machine.

6.      The generated text is printed to the console, confirming the local LLM is working.

 

Result

 

A local LLM environment was successfully configured using Python 3.12.7, PyTorch and Hugging Face Transformers, and a test prompt was executed locally without using any external API, confirming offline model access.