Summary of Learning in Digital Image Processing
Digital Image Processing is one of the most influential specialized courses I took during my undergraduate studies. Professor Li Zhu systematically taught traditional image processing methods and closely integrated theoretical derivations with code practice and engineering case studies, enabling me to develop a comprehensive understanding from problem definition to algorithm implementation. Through the study and practice of methods such as connected component analysis, SIFT/HOG, I further realized that in the era of AI-Coding, the core of engineering capability lies in technical route design, strategy modeling, and result validation, rather than merely code implementation. Compared with platform-based courses that rely on specialized hardware, this course allowed rapid iteration in a personal computing environment with timely feedback, which also motivated me to continuously conduct image processing experiments and project validations outside the classroom.
In-Class Traditional Algorithms: Core Framework
The in-class learning followed the main thread of "image representation → basic operations → features and recognition → frequency domain and coding."
| Module | Core In-Class Points | Key Content I Mastered |
|---|---|---|
| Image Fundamentals | Sampling, quantization, coding; binary/grayscale/color images; pixel neighborhood and connected components | Ability to quickly assess the impact of image representation methods on subsequent algorithm complexity and effectiveness |
| Color and Transformation | RGB, HSV/HSI, YCbCr; geometric transformation and interpolation | Understanding why color space separation is commonly performed in compression and detection, and the engineering significance of perspective correction |
| Enhancement and Segmentation | Convolution filtering, median filtering, histogram transformation, Otsu binarization, morphology | Capable of designing preprocessing pipelines based on noise type and target morphology |
| Features and Detection | HOG/SIFT, Canny, least squares method, Hough transform | Able to compare algorithm advantages and disadvantages and perform scene matching, not limited to calling functions |
| Recognition and Coding | kNN/k-means, background subtraction, JPEG pipeline (DCT/quantization/coding) | Established an overall understanding from "feature extraction" to "classification and recognition" and then to "compression and transmission" |
Main knowledge thread of the DIP course:In-class Knowledge: Algorithm Comparison and Methodological Awareness
HOG vs SIFT
HOG (Histogram of Oriented Gradients) is more focused on overall contour description (pedestrian detection), while SIFT is more oriented toward keypoint matching; the latter has strong robustness to rotation and scale.
Least Squares vs. Hough Transform
Least squares offers high accuracy and speed but is sensitive to outliers; Hough transform is more robust and can detect multiple lines, at the cost of higher computational expense.
Convolution Filtering vs. Median Filtering
Convolution is suitable for smoothing random noise; median filtering is more stable for suppressing salt-and-pepper noise and better preserves edges.
Otsu Threshold Segmentation
Automatically selects a threshold by maximizing inter-class variance; it is a high-frequency fundamental method for separating grayscale foreground and background.
Extracurricular Practice: Closed-loop Classroom Exercises (1–8)
Based on the organization of classroom exercises, I completed a full set of practices including object counting, morphological cleaning, connected component filtering, perspective correction, Hough line detection, gamma enhancement, and color object recognition. These exercises helped me develop a stable engineering habit of "first preprocessing, then detection/segmentation, and finally result review."
| Practice Stage | Representative Tasks | Method Pipeline |
|---|---|---|
| Exercise 4-5 | Perspective Correction for Tilted Book Images | Edge Detection + Color Space Masking → Corner Localization → Perspective Transformation to Restore Rectangular View |
| Exercise 6 | Extraction of Main Contour Lines for Workpiece | Canny → Edge Repair → HoughLinesP → Clustering → Least Squares Fitting |
| Exercise 7 | Image Gamma Correction UI | Normalization → Power Transformation → Curve Visualization → Interactive Parameter Adjustment |
| Exercise 8 | Red Bottle Cap Recognition | BGR→HSV → Dual-Range Red Mask → Opening and Closing Operations → Maximum Contour Localization |
Extracurricular Practice: Comprehensive Project (Final Report)
This section organizes the content based on the final comprehensive report of the course, focusing on two representative projects: hand keypoint calibration for piano playing and single-character segmentation of ancient calligraphy manuscripts.
Project 1: Calibration of Hand Keypoint Actions for Piano Performance
This project focuses on single-camera piano performance videos. The objective is to align "audible note events" with "visible hand keypoint actions" on the same timeline, producing a calibrated visualization video and structured data. The core approach is a "video branch + audio branch + fusion judgment" framework: the video branch handles keyboard alignment, key mapping, and hand keypoints; the audio branch extracts note onset and offset events; the fusion branch filters false detections using hand reachability and temporal consistency.
Step 1: Keyboard Calibration and Perspective Normalization
By solving the homography matrix from four corner points, the keyboard region from different camera perspectives is uniformly mapped to a fixed ROI coordinate system.
Step 2: Keyboard Tracking and Hand Mapping
Optical flow and robust estimation are used to suppress frame drift, and 21-point hand keypoints are employed to determine the reachable key range for each frame.
Step 3: Audio Transcription and Onset Event Extraction
The audio track is extracted and transcribed into MIDI, constructing a temporally sorted sequence of note onset and duration events.
Step 4: Audio-Visual Fusion Judgment and Visualization Output
Using "audio candidates + visual reachability constraints," obvious false detections are filtered out, generating key highlighting, hand skeleton overlay, and structured annotation output.
Project 2: Single-Character Segmentation of Ancient Calligraphy Masterpieces (Pioneering Demo)
This project addresses the segmentation of individual characters from extremely large calligraphy masterpieces, adopting an engineering approach primarily based on traditional image algorithms with minimal manual correction. In the report, the workflow starts from sample analysis and failure cases, gradually converging to a stable pipeline: "preprocessing mask → column segmentation → row segmentation → four-step post-processing → result validation."
Step 1: Adaptive Ink Color and Noise Preprocessing
Binarization and connected component filtering are first applied to suppress background noise such as paper texture and water stains, preserving valid character pixels.
Step 2: Red Seal Mask Suppression
Red seals are extracted in HSV color space and suppressed via masking to prevent them from forming "false stroke columns" in projection statistics.
Step 3: Column Segmentation and Row Scanning
The vertical projection minima are used to locate column boundaries, and then horizontal projection is performed on each column to obtain initial row segments.
Step 4: Four-Step Post-Processing (Including Merging Algorithm)
Boundary expansion, gap-driven merging, nearest-merging of abnormal small bounding boxes, and character-spacing-based verification splitting are executed to repair issues such as broken long characters and fragmented bounding boxes.
Function of the Mask: Preprocessing Noise Reduction and Seal Filtering
Merging Algorithm: Nearest-Merging Mechanism for Abnormal Small Bounding Boxes
In the initial draft of row segmentation, some segments may contain "abnormally small bounding boxes." The merging algorithm first identifies these abnormal boxes based on statistical height, then merges them into more appropriate neighboring characters by considering vertical spacing and pixel connectivity. After merging, height constraints are verified to reduce the risk of incorrect merging.
Key Preview (Large Image): Example of Semi-Cursive Script and Final Segmentation Results
Processing Test on My Own Calligraphy Works
Course Outcomes and Application Value
Closing the Loop from Theory to Engineering
Ability to independently convert algorithmic principles into executable workflows, and to perform visualization and error analysis on intermediate steps.
Method Selection Capability
Depending on the quality of the image and the task objectives, the method can be selected based on noise type, morphological features, and real-time requirements.
Interdisciplinary Transferability
The course methodology has been repeatedly validated in competitions, course projects, and independent experiments, demonstrating strong transferability.
Assessment of Long-Term Research Value
I believe that digital image processing holds significant academic and applied value, and I aspire to continue pursuing in-depth research in related directions.