TheHauData Mentoring Program · Track 1 · Lesson 6

Review, Built-in Functions and SQL Optimisation

One-page recap of Lessons 1–5, the built-in functions you will use daily, two optimisation tips, and a function playground. The homework is shared with Lesson 5.

Lesson 6 reviews everything so far, adds the built-in functions that clean and reshape data, and teaches two habits that make queries faster and cheaper. Then play SQL Noir to practise.

Review Lessons 1–5 in one map

Concepts
  • How data is stored: tables, schema, database, warehouse
  • Data types
  • Fact and dimension tables, keys, relationships
Key commands
  • SELECT FROM WHERE
  • GROUP BY HAVING ORDER BY TOP
  • JOIN, UNION, CASE WHEN
Complex logic
  • Subquery and CTE
  • Window functions
  • Built-in functions (today)

Execution order to remember: FROM/JOIN → WHERE → GROUP BY → HAVING → SELECT → ORDER BY → TOP.

Convert CAST and COALESCE

  • CAST(x AS type) changes a data type. You used it all through Lesson 3 to join keys of different types, and in Lesson 4 to build percentages.
  • COALESCE(a, b, c) returns the first value that is not NULL. Typical use: show 'Cash' when there is no card type, or 0 when a number is missing.

Dates Add, extract, compare

FunctionDoes
DATEADD(day, 7, d)add 7 days → 2023-05-12
YEAR / MONTH / DAY(d)extract a part
DATEPART(part, d)year, quarter, month, weekday ...
DATEDIFF(day, a, b)days between two dates
DATE_FORMAT / FORMATturn a date or number into text in a given pattern

Which exact names exist depends on the dialect. The course database is T-SQL.

Text Clean and cut strings

FunctionExampleResult
LENLEN('banana')6
TRIMTRIM(' banana ')banana
REPLACEREPLACE('banana','a','e')benene
LEFT / RIGHTRIGHT('banana',3)ana
SUBSTRINGSUBSTRING('banana',2,3)ana
CONCATCONCAT('apple','banana')applebanana
FORMATFORMAT(12345.6789,'#,##0.00')12,345.68

Speed Two optimisation tips

  1. Remove extra processing. Select only the columns you need. Skip DISTINCT, ORDER BY and extra joins that the result does not need.
  2. Reduce the dataset early. Filter in WHERE, or inside a subquery or CTE, before joining and grouping. Fewer rows enter every later step.

Lesson 4's example: orders have 1M rows × 10 columns, products 100 rows × 100 columns. Trim both to what you use before the JOIN.

Practice SQL Noir

The slides end with a detective-style SQL game: sqlnoir.com. Solve cases using JOIN, GROUP BY, CTE and window functions.

Homework Where the Lesson 5–6 homework uses these functions

FunctionWhere it appears in the answer key
CASTJoining sub-category and category keys (HW 2, 4, 5, 7, 8, 9) and TRY_CAST in HW 7
COALESCE / CASECash when no card (HW 9)
YEAR / MONTHYearly and monthly grouping (HW 3, 4, 5, 7)

Open the Lesson 5 guide for the 10 homework solutions.

Watch out Mistakes with built-in functions

  • A function on a column blocks filtering shortcuts. WHERE YEAR(OrderDate) = 2012 is fine for learning. On huge tables a range (OrderDate >= '2012-01-01' AND OrderDate < '2013-01-01') is usually faster because it can use an index.
  • Dialect differences. LEN and DATEADD are T-SQL. MySQL and BigQuery use LENGTH, DATE_ADD or DATE_FORMAT. Check the dialect before copying code.
  • Two slide typos. REPLACE('banana','a','e') returns benene (lower case), not "Benene". SUBSTRING('banana', 2, 3) starts at position 2 (the first a) and returns ana, not "nan". Positions count from 1. Try both in the playground.
  • Weekday numbers depend on a server setting. Test DATEPART(weekday, ...) on a known date.
Source: Lesson 6.pdf (Quoc-Hau Le, TheHauData).
Lesson 6 has no class-practice or homework file of its own: its homework is the same set as Lesson 5 (HW_Ref_solution_L056.sql, stored in both folders). The tasks below are new practice written for this guide, not from the answer key. Results come from running equivalent queries on the CSV files.

Playground Try a function on your own input

Each card runs the function in your browser and shows the SQL you would write. Edit the values.