1. Objectives
- Explain the hybrid learning of ANFIS: least squares for the rule formulas, gradient descent for the membership functions.
- Compute a numerical gradient of the error with respect to a membership-function parameter.
- Train the irrigation ANFIS and read learning and test error curves to choose the number of epochs.
- Compare the tuned ANFIS with P11a, a linear formula and an MLPRegressor network.
2. Word Bank
Word | Simple meaning | Real-world example |
|---|---|---|
Hybrid learning | Two methods each epoch: least squares (forward) and gradient descent (backward) | Jang's ANFIS training |
Gradient | The slope of the error: how much it changes when a parameter changes a little | +0.146 for the dry centre |
Numerical gradient | Slope estimated as (error after a tiny change - error before) / tiny change | h = 0.001 |
Learning rate | How big a step gradient descent takes | lr = 2 |
Epoch | One full pass of learning over the data | 40 epochs |
Overfitting | Learning error keeps falling while test error stops falling or rises | choose the best epoch |
3. Background
What P11a left fixed
In P11a the bells were our guess: dry and wet cross at 35 % moisture. But the farm data bend sharply around 25-30 %. Least squares chose the best rule formulas FOR THOSE bells (test RMSE 5.43 min). Today we move the bells too.
Gradient descent
Like walking downhill in fog: feel the slope under your feet, take a step the other way. For each parameter (centre c and width a of each bell), nudge it by h = 0.001, see how the error changes, and move it by - lr x slope. We use a numerical slope so no calculus is needed; MATLAB's anfis() uses the exact derivative, which gives the same direction.
One epoch of hybrid learning
(1) forward pass: least squares for the 12 rule numbers (P11a fit); (2) backward pass: gradient step on the 8 bell numbers; repeat. Watch both errors - learning RMSE always falls, but only the test RMSE says whether the model is really improving.
Data files for this lab
irrigation.csv
moisture,temperature,minutes 16.4,19.3,77.1 35.0,28.3,26.2 40.1,38.6,11.5 11.4,41.8,111.6 17.4,37.8,101.9 56.4,16.1,7.2 13.5,25.8,90.7 16.5,19.9,82.9 57.4,45.0,11.0 41.1,19.3,13.2 ... (100 data rows in total - copy the full file from the lab folder) |
4. Time Plan (50 minutes)
Minutes | Activity | Part |
|---|---|---|
0-5 | Background: hybrid learning, gradient descent | 1-3 |
5-10 | Guided warm-up: the bells vs the data; one gradient | Part A |
10-13 | Predict the output | Part B |
13-17 | Bug hunt | Part C |
17-47 | Graded tasks 1-3 | Part D |
47-50 | Wrap-up; post-lab questions set as homework | Part E |
Part A - Guided Warm-up (type, run, observe)
A1. Where do the P11a bells cross?
Plot the two moisture bells over the data. The crossover of dry and wet is where the model 'changes its mind'.
import numpy as np import pandas as pd df = pd.read_csv("irrigation.csv") X = df[["moisture", "temperature"]].to_numpy() y = df["minutes"].to_numpy() X_tr, y_tr, X_te, y_te = X[:70], y[:70], X[70:], y[70:] # 70 readings to learn, 30 to test rmse = lambda pred, true: float(np.sqrt(np.mean((pred - true) ** 2))) def gbell(x, a, b, c): return 1 / (1 + np.abs((x - c) / a) ** (2 * b)) MF = {"moisture": {"dry": [25, 2, 10], "wet": [25, 2, 60]}, # [a, b, c] for each term "temperature": {"mild": [15, 2, 15], "hot": [15, 2, 45]}} RULES = [("dry", "mild"), ("dry", "hot"), ("wet", "mild"), ("wet", "hot")] def strengths(X, MF): """Layers 1-3: degrees, product AND, normalise so each row adds to 1.""" w = np.array([gbell(X[:, 0], *MF["moisture"][m]) * gbell(X[:, 1], *MF["temperature"][t]) for m, t in RULES]).T return w / w.sum(axis=1, keepdims=True) def design(X, wn): """Layer 4 inputs: for each rule k the columns wn_k*moisture, wn_k*temperature, wn_k.""" Xe = np.c_[X, np.ones(len(X))] return np.hstack([wn[:, [k]] * Xe for k in range(len(RULES))]) def fit(X, y, MF): p, *_ = np.linalg.lstsq(design(X, strengths(X, MF)), y, rcond=None) # least squares return p def predict(X, MF, p): return design(X, strengths(X, MF)) @ p # layer 5: sum import matplotlib.pyplot as plt p = fit(X_tr, y_tr, MF) print(f"P11a model: learning RMSE {rmse(predict(X_tr, MF, p), y_tr):.2f}, test RMSE {rmse(predict(X_te, MF, p), y_te):.2f} min") m = np.linspace(10, 60, 201) cross = m[np.argmin(abs(gbell(m, *MF["moisture"]["dry"]) - gbell(m, *MF["moisture"]["wet"])))] print(f"dry and wet cross at moisture {cross:.1f} %") fig, ax = plt.subplots(figsize=(7, 2.6)); ax.scatter(df.moisture, df.minutes, s=10, c="grey") ax.set_xlabel("Soil moisture (%)"); ax.set_ylabel("Minutes"); ax2 = ax.twinx() for term, prm in MF["moisture"].items(): ax2.plot(m, gbell(m, *prm), lw=2, label=term) ax2.set_ylabel("Degree"); ax2.legend(loc="center right"); plt.tight_layout(); plt.savefig("bells.png", dpi=90) print("Saved bells.png") |
Output P11a model: learning RMSE 3.61, test RMSE 5.43 min dry and wet cross at moisture 35.0 % Saved bells.png |
A2. One numerical gradient
How does the learning error change if the centre of 'dry' moves right by 0.001?
# ... set-up: load the irrigation data (as shown earlier) ... # ... set-up: the ANFIS model (membership functions, rules, layers) (as shown earlier) ... def fit(X, y, MF): p, *_ = np.linalg.lstsq(design(X, strengths(X, MF)), y, rcond=None) # least squares return p def predict(X, MF, p): return design(X, strengths(X, MF)) @ p # layer 5: sum import copy def loss(MF): p = fit(X_tr, y_tr, MF); return rmse(predict(X_tr, MF, p), y_tr) h = 1e-3 before = loss(MF) M2 = copy.deepcopy(MF); M2["moisture"]["dry"][2] += h after = loss(M2) print(f"error before {before:.6f}, after {after:.6f}") print(f"slope = {(after - before) / h:.3f} -> move the dry centre LEFT (downhill)") |
Output error before 3.614013, after 3.614159 slope = 0.146 -> move the dry centre LEFT (downhill) |
Part B - Predict the Output
Write your prediction BEFORE running the code. Then run it and correct yourself in a different colour.
B1.
c, slope, lr = 10.0, 0.146, 2.0 c = c - lr * slope print(round(c, 3)) |
My prediction |
B2.
f = lambda x: x ** 2 x, h = 3, 0.001 print(round((f(x + h) - f(x)) / h, 3)) |
My prediction |
B3.
MF = {"dry": [25, 2, 10]} M2 = MF M2["dry"][2] += 1 print(MF["dry"][2]) |
My prediction |
Part C - Bug Hunt
Each program has ONE bug. Some crash with an error message; others run but give a wrong answer (silent logic error - the dangerous kind). Find the bug, explain it and write the fix.
C1.
# ... set-up: load the irrigation data (as shown earlier) ... # ... set-up: the ANFIS model (membership functions, rules, layers) (as shown earlier) ... def fit(X, y, MF): p, *_ = np.linalg.lstsq(design(X, strengths(X, MF)), y, rcond=None) # least squares return p def predict(X, MF, p): return design(X, strengths(X, MF)) @ p # layer 5: sum import copy PARAMS = [(v, term, j) for v in MF for term in MF[v] for j in (0, 2)] # tune a (width) and c (centre); keep b = 2 def loss(MF): p = fit(X_tr, y_tr, MF) # forward pass: best rule formulas for these MFs return rmse(predict(X_tr, MF, p), y_tr) def gradients(MF, h=1e-3): base = loss(MF); g = {} for v, term, j in PARAMS: M2 = copy.deepcopy(MF); M2[v][term][j] += h g[(v, term, j)] = (loss(M2) - base) / h # slope = change in error / small change return base, g M = copy.deepcopy(MF) for ep in range(1, 6): err, g = gradients(M) for (v, term, j), s in g.items(): M[v][term][j] += 2.0 * s print(f"epoch {ep}: learning RMSE {err:.3f}") |
It runs and prints epoch 1: learning RMSE 3.614 epoch 2: learning RMSE 3.717 epoch 3: learning RMSE 3.811 epoch 4: learning RMSE 3.898 epoch 5: learning RMSE 3.977 |
Bug, why it happens, and the fixed line(s) |
C2.
MF = {"moisture": {"dry": (25, 2, 10), "wet": (25, 2, 60)}} MF["moisture"]["dry"][2] -= 0.3 print(MF) |
Bug, why it happens, and the fixed line(s) |
C3.
# ... set-up: load the irrigation data (as shown earlier) ... # ... set-up: the ANFIS model (membership functions, rules, layers) (as shown earlier) ... def fit(X, y, MF): p, *_ = np.linalg.lstsq(design(X, strengths(X, MF)), y, rcond=None) # least squares return p def predict(X, MF, p): return design(X, strengths(X, MF)) @ p # layer 5: sum # ... set-up: loss() and the numerical gradient (as shown earlier) ... M = copy.deepcopy(MF) for ep in range(1, 9): err, g = gradients(M) for (v, term, j), s in g.items(): M[v][term][j] -= 60 * s print(f"epoch {ep}: {err:.2f}", end=" ") |
It runs and prints epoch 1: 3.61 epoch 2: 3.35 epoch 3: 3.53 epoch 4: 5.03 epoch 5: 4.97 epoch 6: 3.42 epoch 7: 2.45 epoch 8: 3.02 |
Bug, why it happens, and the fixed line(s) |
Part D - Graded Tasks (10 marks)
Task | Level | Title | Marks |
|---|---|---|---|
T1 | Easy | The gradient of every bell parameter | 3 |
T2 | Medium | Hybrid training and error curves | 3 |
T3 | Challenge | Was it worth it? | 4 |
Total | 10 |
Task 1 (Easy, 3 marks): The gradient of every bell parameter
Write loss(MF) (fit the rule formulas, return the learning RMSE) and gradients(MF) that returns the numerical slope for the width a and centre c of all four bells (8 parameters, h = 0.001, use copy.deepcopy). Print the 8 slopes for the P11a bells. Which bells matter most, and why are the temperature slopes so small?
Sample run (your output must match):
Output learning RMSE 3.614 moisture dry a slope +0.114 moisture dry c slope +0.146 moisture wet a slope -0.077 moisture wet c slope +0.112 temperature mild a slope +0.001 temperature mild c slope -0.000 temperature hot a slope +0.003 temperature hot c slope -0.003 |
Your code (or attach the file) |
Task 2 (Medium, 3 marks): Hybrid training and error curves
Write train(MF, epochs=40, lr=2.0): each epoch take one gradient step on the 8 bell parameters, then refit the rule formulas, and record the learning and test RMSE. Print every 10th epoch, plot both curves (task2.png), and give the epoch with the lowest test RMSE. Compare with P11a (test 5.43 min).
Sample run (your output must match):
Output epoch 10: learning RMSE 2.20, test RMSE 3.14 min epoch 20: learning RMSE 1.72, test RMSE 2.10 min epoch 30: learning RMSE 1.72, test RMSE 2.09 min epoch 40: learning RMSE 1.72, test RMSE 2.10 min lowest test RMSE 2.09 min at epoch 26 (P11a: 5.43) Saved task2.png |
Your code (or attach the file) |
Task 3 (Challenge, 4 marks): Was it worth it?
After 40 epochs, compare the test RMSE and the number of learned parameters of: the P11a ANFIS (12), the tuned ANFIS (12 + 8), the linear formula (3) and an MLPRegressor with 10 hidden neurons in a StandardScaler pipeline (random_state=1, max_iter=5000). Plot the moisture bells before and after tuning (task3.png) and say where the dry/wet crossover moved.
Sample run (your output must match):
Output linear formula 3 parameters test RMSE 11.31 min MLPRegressor 41 parameters test RMSE 6.98 min ANFIS (P11a) 12 parameters test RMSE 5.43 min ANFIS tuned 20 parameters test RMSE 2.10 min dry/wet crossover: before 35.0 %, after 26.8 % Saved task3.png |
Your code (or attach the file) |
Part E - Post-Lab Questions (finish as homework if time runs out)
E1. What are the two passes of ANFIS hybrid learning, and which parameters does each learn?
Your answer |
E2. Why must gradient descent subtract the slope?
Your answer |
E3. How do the learning and test curves help choose the number of epochs?
Your answer |
Marking Rubric (applied to Part D)
Criterion | Full | Partial | None |
|---|---|---|---|
Code runs without errors | Runs fully | Small fixable error | Does not run |
Correct loss, numerical gradient (deepcopy) and descent step | All correct | One slip | Several wrong |
Training loop, printed epochs and error-curve plot | All present | Some missing | None |
Comparison table, bell plot and explanation | Clear and correct | Partly correct | Missing |
Submission
- Make a folder RollNo_P11b with task1.py, task2.py, task3.py (or P11b.ipynb), task2.png and task3.png.
- Upload the zipped folder to the LMS / Google Classroom by the end of the day.
- Copy Task 2 code, output and plot into your Lab Book (Practical 11b page).
- Keep your tuned model - P13 (Deploying a FIS) saves a fuzzy model to a file.