spark-optimization
Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines.
Install
npx skills add wshobson/agents@spark-optimizationRuns in your terminal. Adds the skill globally for Claude Code, Cursor, Codex and others; add -g -y to skip prompts.
What it does
Apache Spark Optimization Production patterns for optimizing Apache Spark jobs including partitioning strategies, memory management, shuffle optimization, and performance tuning. When to Use This Skill - Optimizing slow Spark jobs - Tuning memory and executor configuration - Implementing efficient partitioning strategies - Debugging Spark performance issues - Scaling Spark pipelines for large datasets - Reducing shuffle and data skew Core Concepts 1. Spark Execution Model 2. Key Performance Factors Factor Impact Solution ----------------- --------------------- -----------------------------…
Excerpt from the skill's own SKILL.md. Read the full file on GitHub before installing: skills run with your agent's permissions.
View source on GitHubBefore you install
Skills are plain text instructions the agent follows, sometimes with scripts. Check the source, prefer repositories with many installs and stars, and read any script it ships.
Categories
Related skills
- tailwind-design-system
wshobson/agents
Build scalable design systems with Tailwind CSS v4, design tokens, component libraries, and responsive patterns. Use when creating component libraries, implementing design systems, or standardizing UI patterns.
62K installs 39KDesign & UIReact & Next.js - brand-landingpage
wshobson/agents
>
38K installs 39KDesign & UIMarketingResearch - nextjs-app-router-patterns
wshobson/agents
Master Next.js 14+ App Router with Server Components, streaming, parallel routes, and advanced data fetching. Use when building Next.js applications, implementing SSR/SSG, or optimizing React Server Components.
29K installs 39KReact & Next.jsData - code-review-excellence
wshobson/agents
Master effective code review practices to provide constructive feedback, catch bugs early, and foster knowledge sharing while maintaining team morale. Use when reviewing pull requests, establishing review standards, or mentoring developers.
28K installs 39KCode review - api-design-principles
wshobson/agents
Master REST and GraphQL API design principles to build intuitive, scalable, and maintainable APIs that delight developers. Use when designing new APIs, reviewing API specifications, or establishing API design standards.
28K installs 39KDesign & UITestingDocumentation - postgresql-table-design
wshobson/agents
Use this skill when designing or reviewing a PostgreSQL-specific schema. Covers best-practices, data types, indexing, constraints, performance patterns, and advanced features
25K installs 39KDesign & UIDataProductivity