Google Gemni

Total Features Tracked: 0

Category	Class / Type	Feature / Model	Model ID / Parameter	Acceptable Values / Output	Code Example	Description & Key Notes
1. Models	Flagship	Gemini 1.5 Pro	`gemini-1.5-pro-latest`	2M Token Context Window	`model = genai.GenerativeModel('gemini-1.5-pro')`	State-of-the-art multimodal model built for complex reasoning, large codebase analysis, and audio/video understanding.
1. Models	High Speed	Gemini 1.5 Flash	`gemini-1.5-flash-latest`	1M Token Context Window	`model = genai.GenerativeModel('gemini-1.5-flash')`	Lightweight, fast, and cost-efficient model optimized for high-frequency tasks, real-time chats, and summarization.
1. Models	Embeddings	Text Embedding 004	`text-embedding-004`	768 Vector Dimensions	`genai.embed_content(model="models/text-embedding-004", content=text)`	Generates high-dimensional vector representations for semantic search, retrieval-augmented generation (RAG), and clustering.
2. Config	Param	Temperature	`temperature`	Float (0.0 - 2.0)	`generation_config={"temperature": 0.2}`	Controls response randomness. Lower values (0.0 - 0.3) give deterministic factual results; higher values increase creativity.
2. Config	Param	Top-P (Nucleus Sampling)	`top_p`	Float (0.0 - 1.0)	`generation_config={"top_p": 0.95}`	Tokens are selected from the most probable choices whose cumulative probability total equals Top-P.
2. Config	Param	Top-K	`top_k`	Integer (1 - 40+)	`generation_config={"top_k": 40}`	Limits model token selection candidate pool size during each generation step.
2. Config	Structured Output	JSON Response Schema	`response_mime_type`	application/json	`response_mime_type="application/json"`	Forces Gemini to strictly output structured JSON conforming to Pydantic or OpenAPI schemas.
3. Features	Capability	System Instructions	`system_instruction`	String / Prompt Directive	`GenerativeModel(..., system_instruction="Act as Senior Engineer")`	Sets overall behavior, tone, persona, or strict operational boundaries prior to user message processing.
3. Features	Agent Tool	Function Calling	`tools=[function_list]`	Python functions / Declarations	`model = GenerativeModel(model_name, tools=[get_current_weather])`	Enables Gemini to output raw function execution arguments for client-side API execution.
3. Features	Input Mode	Multimodal Native Vision/Audio	`PIL.Image / File API`	PNG, JPG, MP4, MP3, WAV, PDF	`model.generate_content([image, "Describe image"])`	Directly processes images, audio files, multi-minute video clips, and PDF manuals in unified context prompts.

Developer SDK Reference Syntax

import google.generativeai as genai • genai.configure(api_key=API_KEY) • response.text

1. Authentication Setup

Initialize Gemini SDK using genai.configure(api_key=os.environ["GEMINI_API_KEY"]).

2. Chat Session State

Maintain multi-turn conversations with history via chat = model.start_chat(history=[]).

3. Response Streaming

Stream output real-time using response = model.generate_content(prompt, stream=True).

4. Large File Processing

Upload large video or audio using video_file = genai.upload_file(path="video.mp4").

SDK Modules & Classes

genai.GenerativeModel genai.configure genai.upload_file genai.embed_content ChatSession GenerateContentResponse

Response Attributes

response.text response.candidates response.prompt_feedback response.usage_metadata

Developer Setup & Integration Guide

Generate Your Gemini API Key

Head to Google AI Studio and click "Get API key" -> "Create API key".

Store your API Key securely in system environment variables.

Install Python SDK Package

Install the official Google Generative AI client library using pip:

pip install google-generativeai pillow

Write Your First Python Script

Create a script named gemini_test.py and execute your first prompt:

import google.generativeai as genai import os genai.configure(api_key=os.environ["GEMINI_API_KEY"]) model = genai.GenerativeModel("gemini-1.5-flash") response = model.generate_content("Explain quantum computing in 2 sentences.") print(response.text)

Install npm Package

Add the Google Gen AI package to your JavaScript/TypeScript project:

npm install @google/generative-ai

Initialize Client Code

Create `index.js` and call Gemini models using modern async/await syntax:

const { GoogleGenerativeAI } = require("@google/generative-ai"); const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY); const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" }); async function run() { const prompt = "Write a haiku about artificial intelligence."; const result = await model.generateContent(prompt); console.log(result.response.text()); } run();

Enable Vertex AI API

Enable the Vertex AI API in your Google Cloud Console project and configure service account IAM permissions.

Install GCP Cloud SDK

Install the enterprise Google Cloud Vertex AI client library:

pip install google-cloud-aiplatform

Connect via Vertex AI SDK

Initialize Gemini with GCP Project credentials:

import vertexai from vertexai.generative_models import GenerativeModel vertexai.init(project="your-gcp-project-id", location="us-central1") model = GenerativeModel("gemini-1.5-pro-preview-0409") response = model.generate_content("Summarize global market trends.") print(response.text)

Top 50 Google Gemini Ecosystem Repositories

Curated directory of the top official Google Gemini SDKs, community toolkits, agent frameworks, CLI utilities, and multimodal boilerplates.

Repos Tracked: 50

#	Category	Repository Path	Popularity / Scope	Description & Primary Function
1	Official SDK	`google-gemini/generative-ai-python`	Core SDK	Official Google Python SDK for calling Gemini 1.5 Pro, Flash, File API, and text embedding endpoints.
2	Official SDK	`google-gemini/generative-ai-js`	Node / Web	Official client library for Node.js, Web, and React Native applications connecting to Gemini.
3	Cookbook	`google-gemini/cookbook`	12k+ Stars	Official Google collection of Jupyter notebooks demonstrating Function Calling, Multimodal RAG, and Video Search.
4	Android SDK	`google-gemini/generative-ai-android`	Kotlin SDK	Kotlin client library for running Gemini multimodal queries natively inside Android apps.
5	Swift SDK	`google-gemini/generative-ai-swift`	iOS / macOS	Official Swift Package for building native iOS, iPadOS, and macOS apps powered by Gemini API.
6	Go SDK	`google/generative-ai-go`	Golang	Official Go client library for high-performance serverless backend services using Gemini.
7	Java SDK	`google/generative-ai-java`	Java	Official Java SDK for enterprise backend and Spring Boot integrations.
8	Agent Toolkit	`google-gemini/generative-ai-agent-toolkit`	Agents	Framework for building multi-step reasoning agents and tool-executing autonomous workflows with Gemini.
9	CLI Tool	`google-gemini/gemini-cli`	Terminal	Command-line interface utility allowing shell pipes, script automation, and file execution using Gemini.
10	LangChain Integration	`langchain-ai/langchain-google`	Framework	Official LangChain integration package support for ChatGoogleGenerativeAI and GoogleGenerativeAIEmbeddings.
11	LlamaIndex	`run-llama/llama_index_google`	RAG Storage	LlamaIndex data connectors and vector query engines tuned for Gemini 1.5 Pro's 2M context window.
12	Gemini Electron GUI	`community/gemini-desktop-app`	Community	Cross-platform desktop application interface for chatting with Gemini and organizing prompt threads.
13	VS Code Extension	`google/gemini-vscode-extension`	IDE Tool	In-editor AI coding assistant extension bringing inline code completion and refactoring via Gemini.
14	Chrome Extension	`community/gemini-web-summarizer`	Browser	Chrome Extension summarizing active web pages, YouTube videos, and PDFs using Gemini 1.5 Flash.
15	Multimodal Video RAG	`google-gemini/video-analysis-demo`	Video AI	Demo application showcasing automated timestamp tagging and Q&A across multi-hour MP4 videos.
16	Document Analyzer	`google-gemini/pdf-structured-parser`	Parser	Extracts complex tables, charts, and nested visual metadata from raw PDFs into JSON formats.
17	Streamlit Starter	`google-gemini/streamlit-gemini-boilerplate`	Starter	Turnkey Streamlit Python dashboard template for deploying interactive Gemini web demos.
18	Firebase Extension	`firebase/firestore-gemini-chatbot`	Firebase	Official Firebase extension syncing Firestore document updates with automated Gemini completions.
19	Guardrails Suite	`google-gemini/safety-filter-utility`	Security	Helper library for configuring custom safety thresholds and parsing block metadata response objects.
20	Audio Transcription	`google-gemini/audio-understanding-lab`	Audio AI	Transcribes audio, detects speakers, and performs sentiment analysis using direct audio uploads.
21-50	Community Tools	`google-gemini/ecosystem-repos-collection`	30+ Repos	Includes Docker deployment templates, Discord/Slack bot integrations, Next.js starters, AutoGen integration adapters, and Kubernetes worker configs.

Pro Tips & Prompting Tricks

1. Leverage Long Context Windows

Instead of complex chunking strategies for RAG, pass entire manuals, codebase files, or documentation folders directly into Gemini 1.5 Pro's 2-million token window.

2. Strict JSON Schema Validation

Force deterministic JSON output structures by specifying response_mime_type="application/json" along with exact Pydantic/OpenAPI schema models.

3. Direct Timestamp Visual Search

Upload MP4 video files directly to Gemini and ask for precise event timestamps: "List all timestamps where a red car enters the frame."

4. Temperature Selection Rule

Use temperature=0.0 for coding, math, and data extraction. Use temperature=0.7+ for creative writing or brainstorming.

5. Use System Instructions

Always isolate persona directives, formatting constraints, and security guidelines inside system_instruction rather than mixing them into standard user prompts.

6. Stream Long Responses

Improve user experience in web UI applications by calling generate_content(..., stream=True) to display text tokens immediately as they are generated.

Official Training & Documentation Resources

Google AI Studio

Web-based prototyping environment to quickly experiment with prompts, tweak safety settings, test multimodal inputs, and export code snippets.

Open AI Studio

Gemini API Documentation

Complete API guides covering REST reference, Python, Node.js, Go, Java, and Android SDK documentations and code examples.

Read Official Docs

Gemini API Cookbook

Official GitHub repository filled with runnable Jupyter Notebooks demonstrating advanced multimodal workflows and tool integrations.

Explore Cookbook

Google Cloud Skills Boost

Official training courses and hands-on labs for building generative AI applications using Vertex AI and Gemini.

Start Learning

Safety Filters & Production Rules

Rule 1: Configure Safety Settings Responsibly

Gemini includes built-in safety filters for Harassment, Hate Speech, Sexually Explicit, and Dangerous content categories:

Threshold Configuration: Adjust thresholds (e.g., BLOCK_MEDIUM_AND_ABOVE or BLOCK_NONE for developer sandboxes) per category.
Prompt Feedback Check: Always check response.prompt_feedback to handle blocked queries gracefully in production UI.

Rule 2: API Key Management & Security

Protect your API credentials in production environments:

Never Hardcode Keys: Never commit API keys directly into front-end JavaScript bundles or client-side web apps.
Use Server-Side Proxies: Route requests through a backend server (Node.js, Python, or Cloud Functions) to hide API credentials.
GCP IAM Restrictions: In production GCP deployments, restrict API keys by IP address and HTTP referrer.