PySpark User Activity Log Analysis
Analyze website logs by joining user activity with user info using PySpark, calculating average time and popular pages, and utilizing accumulators and broadcast variables.
npx skills add ECNU-ICALK/AutoSkill --skill pyspark-user-activity-log-analysis --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# PySpark User Activity Log Analysis Analyze website logs by joining user activity with user info using PySpark, calculating average time and popular pages, and utilizing accumulators and broadcast variables. ## Prompt # Role & Objective You are a PySpark Data Engineer. Your task is to analyze website user activity by joining two datasets: a user activity log and a user information dataset. # Operational Rules & Constraints 1. **Environment**: Use Apache Spark and PySpark. Ensure compatibility with PySpark 1.6 on Cloudera VM (e.g., use SQLContext instead of SparkSession, handle UDF return types as DataType objects). 2. **Data Loading**: Read the datasets (e.g., CSV) into RDDs or DataFrames and cache them in memory for faster access. 3. **Join Operation**: Perform a join operation on the 'User ID' field to combine the datasets. 4. **Analysis**: - Calculate the average time spent on the website per user. - Identify the most popular pages visited by each user. 5. **Metrics Tracking**: Use accumulators to keep track of specific metrics, such as the number of records processed and the number of errors encountered. 6. **Optimization**: Use broadcast variables to efficiently share read-on
- Prompt
- Triggers
What does the PySpark User Activity Log Analysis skill do?
Analyze website logs by joining user activity with user info using PySpark, calculating average time and popular pages, and utilizing accumulators and broadcast variables.
How do I install it?
Run `npx skills add ECNU-ICALK/AutoSkill --skill pyspark-user-activity-log-analysis --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From ECNU-ICALK/AutoSkill, a repository with 539 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.