Home > Study > Linux > Day3 : Perceptron

Day3 : Perceptron
Study Language

๐Ÿ“Œ Perceptron๋ž€?


ํผ์…‰ํŠธ๋ก (Perceptron)์€ ์ƒ๋ฌผํ•™์  ๋‰ด๋Ÿฐ์„ ์ˆ˜ํ•™์ ์œผ๋กœ ๋ชจ๋ธ๋งํ•œ ์ธ๊ณต ๋‰ด๋Ÿฐ ๋ชจ๋ธ๋กœ, ์—ฌ๋Ÿฌ ์ž…๋ ฅ ์‹ ํ˜ธ๋ฅผ ๋ฐ›์•„ ๊ฐ ์ž…๋ ฅ์— ๋Œ€ํ•œ ๊ฐ€์ค‘์น˜(Weight)๋ฅผ ๊ณฑํ•œ ํ›„, ์ด๋“ค์˜ ๊ฐ€์ค‘ํ•ฉ(Weighted Sum)์„ ๊ณ„์‚ฐํ•˜๊ณ , ํ™œ์„ฑํ™” ํ•จ์ˆ˜(Activation Function)๋ฅผ ํ†ตํ•ด ์ตœ์ข… ์ถœ๋ ฅ์„ ๊ฒฐ์ •ํ•˜๋Š” ๊ตฌ์กฐ์ด๋‹ค.


๐Ÿ”ง ๊ตฌ์กฐ (Perceptron Structure)

์ž…๋ ฅ(x) โ†’ ๊ฐ€์ค‘์น˜(w) โ†’ ๊ฐ€์ค‘ํ•ฉ(โˆ‘) โ†’ ํ™œ์„ฑํ™” ํ•จ์ˆ˜(f) โ†’ ์ถœ๋ ฅ(y)
  • ์ž…๋ ฅ (Input): AND, OR ๋“ฑ ๋…ผ๋ฆฌ ์—ฐ์‚ฐ์„ ์œ„ํ•œ ์ž…๋ ฅ ์‹ ํ˜ธ.
  • ๊ฐ€์ค‘์น˜ (Weight): ์ž…๋ ฅ ์‹ ํ˜ธ์˜ ์ค‘์š”๋„๋ฅผ ๊ฒฐ์ •ํ•˜๋ฉฐ, ํ•™์Šต์„ ํ†ตํ•ด ์กฐ์ •๋จ.
  • ๊ฐ€์ค‘ํ•ฉ (Weighted Sum): ๊ฐ ์ž…๋ ฅ๊ณผ ๊ทธ์— ๋Œ€์‘ํ•˜๋Š” ๊ฐ€์ค‘์น˜์˜ ๊ณฑ์„ ๋ชจ๋‘ ๋”ํ•œ ๊ฐ’.
  • ํ™œ์„ฑํ™” ํ•จ์ˆ˜ (Activation Function): ๊ฐ€์ค‘ํ•ฉ์ด ์ž„๊ณ„๊ฐ’์„ ๋„˜์œผ๋ฉด 1, ๋„˜์ง€ ๋ชปํ•˜๋ฉด 0์„ ์ถœ๋ ฅํ•˜๋Š” ํ•จ์ˆ˜. ๋Œ€ํ‘œ์ ์œผ๋กœ ๋‹จ์œ„ ๊ณ„๋‹จ ํ•จ์ˆ˜ ์‚ฌ์šฉ.
  • ์ถœ๋ ฅ (Output): ์ตœ์ข… ๊ฒฐ๊ณผ๊ฐ’ (๋ณดํ†ต 0 ๋˜๋Š” 1์˜ ์ด์ง„ ์ถœ๋ ฅ).

๐ŸŽฏ ์š”์•ฝ

  • ํผ์…‰ํŠธ๋ก ์€ ์ด์ง„ ๋ถ„๋ฅ˜ ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•  ์ˆ˜ ์žˆ๋Š” ๊ฐ€์žฅ ๊ธฐ๋ณธ์ ์ธ ์‹ ๊ฒฝ๋ง ๊ตฌ์กฐ์ด๋‹ค.
  • ํ•™์Šต์„ ํ†ตํ•ด ์ž…๋ ฅ ์‹ ํ˜ธ์˜ ์ค‘์š”๋„๋ฅผ ๋‚˜ํƒ€๋‚ด๋Š” ๊ฐ€์ค‘์น˜๊ฐ€ ์กฐ์ •๋œ๋‹ค.
  • ๋‹จ์ธต ํผ์…‰ํŠธ๋ก ์€ ์„ ํ˜• ๋ถ„๋ฆฌ ๊ฐ€๋Šฅํ•œ ๋ฌธ์ œ๋งŒ ํ•ด๊ฒฐํ•  ์ˆ˜ ์žˆ๋‹ค.

๐Ÿ‘จโ€๐Ÿ’ป ์‹ค์Šต


๐Ÿ’ก Code : AND & OR & NAND & XOR Gate Perceptron

# AND & OR & NAND & XOR Gate Perceptron
import numpy as np
import matplotlib.pyplot as plt

class Perceptron:
    def __init__(self, input_size, lr=0.1, epochs=10):
        self.weights = np.zeros(input_size)
        self.bias = 0
        self.lr = lr
        self.epochs = epochs
        self.errors = []

    def activation(self, x):
        return np.where(x > 0, 1, 0)

    def predict(self, x):
        linear_output = np.dot(x, self.weights) + self.bias
        return self.activation(linear_output)

    def train(self, X, y):
        for epoch in range(self.epochs):
            total_error = 0
            for xi, target in zip(X, y):
                prediction = self.predict(xi)
                update = self.lr * (target - prediction)
                self.weights += update * xi
                self.bias += update
                total_error += int(update != 0.0)
            self.errors.append(total_error)
            print(f"Epoch {epoch+1}/{self.epochs}, Errors: {total_error}")

# AND ๊ฒŒ์ดํŠธ ๋ฐ์ดํ„ฐ ๋ฐ ํ•™์Šต
X_and = np.array([[0,0],[0,1],[1,0],[1,1]])
y_and = np.array([0,0,0,1])

print(" AND Gate Training")
ppn_and = Perceptron(input_size=2)
ppn_and.train(X_and, y_and)

print("\n AND Gate Test:")
for x in X_and:
    print(f"Input: {x}, Predicted Output: {ppn_and.predict(x)}")

# OR ๊ฒŒ์ดํŠธ ๋ฐ์ดํ„ฐ ๋ฐ ํ•™์Šต
X_or = np.array([[0,0],[0,1],[1,0],[1,1]])
y_or = np.array([0,1,1,1])

print("\n OR Gate Training")
ppn_or = Perceptron(input_size=2)
ppn_or.train(X_or, y_or)

print("\n OR Gate Test:")
for x in X_or:
    print(f"Input: {x}, Predicted Output: {ppn_or.predict(x)}")

# NAND ๊ฒŒ์ดํŠธ ๋ฐ์ดํ„ฐ ๋ฐ ํ•™์Šต
X_nand = np.array([[0,0],[0,1],[1,0],[1,1]])
y_nand = np.array([1,1,1,0])  # AND์™€ ๋ฐ˜๋Œ€

print("\n NAND Gate Training")
ppn_nand = Perceptron(input_size=2)
ppn_nand.train(X_nand, y_nand)

print("\n NAND Gate Test:")
for x in X_nand:
    print(f"Input: {x}, Predicted Output: {ppn_nand.predict(x)}")

# XOR ๊ฒŒ์ดํŠธ ๋ฐ์ดํ„ฐ ๋ฐ ํ•™์Šต
X_xor = np.array([[0,0],[0,1],[1,0],[1,1]])
y_xor = np.array([0,1,1,0])  # ์„ ํ˜• ๋ถ„๋ฆฌ ๋ถˆ๊ฐ€๋Šฅ

print("\n XOR Gate Training")
ppn_xor = Perceptron(input_size=2)
ppn_xor.train(X_xor, y_xor)

print("\n XOR Gate Test:")
for x in X_xor:
    print(f"Input: {x}, Predicted Output: {ppn_xor.predict(x)}")

โœ… Result : AND & OR & NAND & XOR Gate Perceptron

 AND Gate Training
Epoch 1/10, Errors: 1
Epoch 2/10, Errors: 3
Epoch 3/10, Errors: 3
Epoch 4/10, Errors: 2
Epoch 5/10, Errors: 1
Epoch 6/10, Errors: 0
Epoch 7/10, Errors: 0
Epoch 8/10, Errors: 0
Epoch 9/10, Errors: 0
Epoch 10/10, Errors: 0

 AND Gate Test:
Input: [0 0], Predicted Output: 0
Input: [0 1], Predicted Output: 0
Input: [1 0], Predicted Output: 0
Input: [1 1], Predicted Output: 1

 OR Gate Training
Epoch 1/10, Errors: 1
Epoch 2/10, Errors: 2
Epoch 3/10, Errors: 1
Epoch 4/10, Errors: 0
Epoch 5/10, Errors: 0
Epoch 6/10, Errors: 0
Epoch 7/10, Errors: 0
Epoch 8/10, Errors: 0
Epoch 9/10, Errors: 0
Epoch 10/10, Errors: 0

 OR Gate Test:
Input: [0 0], Predicted Output: 0
Input: [0 1], Predicted Output: 1
Input: [1 0], Predicted Output: 1
Input: [1 1], Predicted Output: 1

 NAND Gate Training
Epoch 1/10, Errors: 2
Epoch 2/10, Errors: 3
Epoch 3/10, Errors: 3
Epoch 4/10, Errors: 0
Epoch 5/10, Errors: 0
Epoch 6/10, Errors: 0
Epoch 7/10, Errors: 0
Epoch 8/10, Errors: 0
Epoch 9/10, Errors: 0
Epoch 10/10, Errors: 0

 NAND Gate Test:
Input: [0 0], Predicted Output: 1
Input: [0 1], Predicted Output: 1
Input: [1 0], Predicted Output: 1
Input: [1 1], Predicted Output: 0

 XOR Gate Training
Epoch 1/10, Errors: 2
Epoch 2/10, Errors: 3
Epoch 3/10, Errors: 4
Epoch 4/10, Errors: 4
Epoch 5/10, Errors: 4
Epoch 6/10, Errors: 4
Epoch 7/10, Errors: 4
Epoch 8/10, Errors: 4
Epoch 9/10, Errors: 4
Epoch 10/10, Errors: 4

 XOR Gate Test:
Input: [0 0], Predicted Output: 1
Input: [0 1], Predicted Output: 1
Input: [1 0], Predicted Output: 0
Input: [1 1], Predicted Output: 0

๐Ÿ’ก Code : ๊ฒฝ๊ณ„ ๊ฒฐ์ • ์‹œ๊ฐํ™” ํ•จ์ˆ˜ (AND, OR, NAND, XOR)

# ๊ฒฝ๊ณ„ ๊ฒฐ์ • ์‹œ๊ฐํ™” ํ•จ์ˆ˜ (AND, OR, NAND, XOR)
from matplotlib.colors import ListedColormap
import matplotlib.pyplot as plt
import numpy as np

def plot_decision_boundary(X, y, model, title='Perceptron Decision Boundary'):
    cmap_light = ListedColormap(['#FFDDDD', '#DDDDFF'])  # ๋ฐฐ๊ฒฝ ์ƒ‰์ƒ
    cmap_bold = ListedColormap(['#FF0000', '#0000FF'])   # ์  ์ƒ‰์ƒ

    h = .02  # mesh grid ๊ฐ„๊ฒฉ

    x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
    y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
                         np.arange(y_min, y_max, h))

    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
    Z = Z.reshape(xx.shape)

    plt.figure(figsize=(6, 5))
    plt.contourf(xx, yy, Z, cmap=cmap_light)
    plt.scatter(X[:, 0], X[:, 1], c=y, cmap=cmap_bold,
                edgecolor='k', s=100, marker='o')
    plt.xlabel('Input 1')
    plt.ylabel('Input 2')
    plt.title(title)
    plt.grid(True)
    plt.show()

# AND ๊ฒŒ์ดํŠธ ๊ฒฐ์ • ๊ฒฝ๊ณ„ ์‹œ๊ฐํ™”
plot_decision_boundary(X_and, y_and, ppn_and, title='AND Gate Decision Boundary')

# OR ๊ฒŒ์ดํŠธ ๊ฒฐ์ • ๊ฒฝ๊ณ„ ์‹œ๊ฐํ™”
plot_decision_boundary(X_or, y_or, ppn_or, title='OR Gate Decision Boundary')

# NAND ๊ฒŒ์ดํŠธ ๊ฒฐ์ • ๊ฒฝ๊ณ„ ์‹œ๊ฐํ™”
plot_decision_boundary(X_nand, y_nand, ppn_nand, title='NAND Gate Decision Boundary')

# XOR ๊ฒŒ์ดํŠธ ๊ฒฐ์ • ๊ฒฝ๊ณ„ ์‹œ๊ฐํ™”
plot_decision_boundary(X_xor, y_xor, ppn_xor, title='XOR Gate Decision Boundary')

โœ… Result : ๊ฒฝ๊ณ„ ๊ฒฐ์ • ์‹œ๊ฐํ™” ํ•จ์ˆ˜ (AND, OR, NAND, XOR)

alt text
alt text
alt text
alt text


๐Ÿ’ก Code : # ์˜ค๋ฅ˜ ์‹œ๊ฐํ™” (AND, OR, NAND, XOR)

# ์˜ค๋ฅ˜ ์‹œ๊ฐํ™” (AND, OR, NAND, XOR)
plt.figure(figsize=(8, 5))
plt.plot(range(1, len(ppn_and.errors) + 1), ppn_and.errors, marker='o', label='AND Gate')
plt.plot(range(1, len(ppn_or.errors) + 1), ppn_or.errors, marker='s', label='OR Gate')
plt.plot(range(1, len(ppn_nand.errors) + 1), ppn_nand.errors, marker='^', label='NAND Gate')
plt.plot(range(1, len(ppn_xor.errors) + 1), ppn_xor.errors, marker='x', label='XOR Gate')
plt.xlabel('Epochs')
plt.ylabel('Number of Errors')
plt.title('Perceptron Learning Error Over Epochs')
plt.legend()
plt.grid(True)
plt.show()

โœ… Result : ์˜ค๋ฅ˜ ์‹œ๊ฐํ™” (AND, OR, NAND, XOR)

alt text

๐Ÿ’ฌ Comment

  • ํผ์…‰ํŠธ๋ก : ์ž…๋ ฅ ๋ฒกํ„ฐ์— ๊ฐ€์ค‘์น˜๋ฅผ ๊ณฑํ•œ ํ•ฉ์ด ๊ธฐ์ค€(0)์„ ๋„˜๋Š”์ง€ ํŒ๋‹จํ•˜๊ณ , ํ•™์Šต ๊ณผ์ •์—์„œ๋Š” ํ‹€๋ฆฐ ๋งŒํผ๋งŒ ์กฐ์ •ํ•˜๋ฉฐ ์„ ํ˜• ๋ถ„๋ฆฌ๋ฅผ ๋ฐฐ์šฐ๋Š” ๊ตฌ์กฐ
  • XOR์€ ์„ ํ˜• ๋ถ„๋ฆฌ ๋ถˆ๊ฐ€๋Šฅํ•œ ๋ฌธ์ œ์ด๊ธฐ ๋•Œ๋ฌธ์—
    ๋‹จ์ธต ํผ์…‰ํŠธ๋ก ์œผ๋กœ๋Š” ํ•ด๊ฒฐํ•  ์ˆ˜ ์—†๋‹ค.
  • ์ด๋ฅผ ํ•ด๊ฒฐํ•˜๋ ค๋ฉด ๋‹ค์ธต ํผ์…‰ํŠธ๋ก (MLP)์ด๋‚˜ ๋น„์„ ํ˜• ๋ณ€ํ™˜์ด ํ•„์š”ํ•˜๋‹ค.

๐Ÿ’ก Code : MLP๋กœ XOR ๋ฌธ์ œ ํ•ด๊ฒฐ

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.colors import ListedColormap

class MultiLayerPerceptron:
    def __init__(self, input_size=2, hidden_size=4, output_size=1, lr=0.5, epochs=1000):
        self.W1 = np.random.uniform(-1, 1, (input_size, hidden_size))
        self.b1 = np.zeros((1, hidden_size))
        self.W2 = np.random.uniform(-1, 1, (hidden_size, output_size))
        self.b2 = np.zeros((1, output_size))
        self.lr = lr
        self.epochs = epochs
        self.losses = []

    def sigmoid(self, x):
        return 1 / (1 + np.exp(-np.clip(x, -250, 250)))

    def sigmoid_derivative(self, x):
        return x * (1 - x)

    def forward(self, X):
        self.z1 = np.dot(X, self.W1) + self.b1
        self.a1 = self.sigmoid(self.z1)
        self.z2 = np.dot(self.a1, self.W2) + self.b2
        self.a2 = self.sigmoid(self.z2)
        return self.a2

    def backward(self, X, y, output):
        m = X.shape[0]
        dZ2 = output - y
        dW2 = (1 / m) * np.dot(self.a1.T, dZ2)
        db2 = (1 / m) * np.sum(dZ2, axis=0, keepdims=True)
        dZ1 = np.dot(dZ2, self.W2.T) * self.sigmoid_derivative(self.a1)
        dW1 = (1 / m) * np.dot(X.T, dZ1)
        db1 = (1 / m) * np.sum(dZ1, axis=0, keepdims=True)

        self.W2 -= self.lr * dW2
        self.b2 -= self.lr * db2
        self.W1 -= self.lr * dW1
        self.b1 -= self.lr * db1

    def train(self, X, y):
        for epoch in range(self.epochs):
            output = self.forward(X)
            loss = np.mean((output - y) ** 2)
            self.losses.append(loss)
            self.backward(X, y, output)
            #if epoch % 200 == 0:
            #    print(f"Epoch {epoch}/{self.epochs}, Loss: {loss:.6f}")

    def predict(self, X):
        output = self.forward(X)
        return (output > 0.5).astype(int)

    def predict_prob(self, X):
        return self.forward(X).ravel()  # ๊ฒฐ์ • ๊ฒฝ๊ณ„์šฉ


# === XOR ๋ฐ์ดํ„ฐ ===
X_xor = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y_xor = np.array([[0], [1], [1], [0]])

# === ํ•™์Šต ===
print("\n=== XOR Gate Multi-Layer Perceptron Training ===")
mlp = MultiLayerPerceptron(input_size=2, hidden_size=2, lr=0.5, epochs=10000)
mlp.train(X_xor, y_xor)

# === ์˜ˆ์ธก ๊ฒฐ๊ณผ ์ถœ๋ ฅ ===
print("\nXOR GATE Test (Multi-Layer Perceptron):")
xor_predictions = mlp.predict(X_xor)
for i, x in enumerate(X_xor):
    predicted = xor_predictions[i][0]
    actual = y_xor[i][0]
    result = "โœ“" if predicted == actual else "โœ—"
    print(f"Input: {x}, Predicted: {predicted}, Actual: {actual}, {result}")


# === ๊ฒฐ์ • ๊ฒฝ๊ณ„ ์‹œ๊ฐํ™” ํ•จ์ˆ˜ ===
def plot_decision_boundary(X, y, model, title="Decision Boundary"):
    cmap_light = ListedColormap(['#FFDDDD', '#DDDDFF'])
    cmap_bold = ListedColormap(['#FF0000', '#0000FF'])

    h = .01
    x_min, x_max = X[:, 0].min() - 0.5, X[:, 0].max() + 0.5
    y_min, y_max = X[:, 1].min() - 0.5, X[:, 1].max() + 0.5
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
                         np.arange(y_min, y_max, h))

    grid = np.c_[xx.ravel(), yy.ravel()]
    Z = model.predict_prob(grid)
    Z = Z.reshape(xx.shape)

    plt.figure(figsize=(6, 5))
    plt.contourf(xx, yy, Z > 0.5, cmap=cmap_light)
    plt.scatter(X[:, 0], X[:, 1], c=y.ravel(), cmap=cmap_bold,
                edgecolor='k', s=120)
    plt.title(title)
    plt.xlabel("Input 1")
    plt.ylabel("Input 2")
    plt.grid(True)
    plt.show()


# === ๊ฒฐ์ • ๊ฒฝ๊ณ„ ์‹œ๊ฐํ™” ===
plot_decision_boundary(X_xor, y_xor, mlp, title="XOR MLP Decision Boundary")

# === ์†์‹ค ๊ณก์„  ์‹œ๊ฐํ™” ===
plt.figure(figsize=(8, 5))
plt.plot(range(mlp.epochs), mlp.losses, color='purple')
plt.title("MLP Training Loss on XOR Problem")
plt.xlabel("Epochs")
plt.ylabel("Mean Squared Error")
plt.grid(True)
plt.show()

โœ… Result : MLP๋กœ XOR ๋ฌธ์ œ ํ•ด๊ฒฐ

=== XOR Gate Multi-Layer Perceptron Training ===

XOR GATE Test (Multi-Layer Perceptron):
Input: [0 0], Predicted: 0, Actual: 0, โœ“
Input: [0 1], Predicted: 1, Actual: 1, โœ“
Input: [1 0], Predicted: 1, Actual: 1, โœ“
Input: [1 1], Predicted: 0, Actual: 0, โœ“

alt text
alt text