Zikang Wu's technical blog and project archive on backend engineering, AI Engineering, indie tools, automation, and long-term writing.https://wzk.icu/enhttps://wzk.icu/en/projects/ai-infra-handbook/https://wzk.icu/en/projects/ai-infra-handbook/AI Infra Handbook is a long-term learning and practice project around AI infrastructure, model serving, vector retrieval, GPU resources, Kubernetes deployment, observability, and cost-aware engineering.Sun, 31 May 2026 00:00:00 GMThttps://wzk.icu/en/projects/ai-voice-module/https://wzk.icu/en/projects/ai-voice-module/An edge-cloud AI voice interaction system experiment for multiple device carriers, exploring how STT, LLM, TTS, agent tools, runtime configuration, and observability form a low-latency maintainable voice pipeline.Sun, 31 May 2026 00:00:00 GMThttps://wzk.icu/en/projects/code-generator/https://wzk.icu/en/projects/code-generator/A code generator practice project for business development workflows, exploring how templates, metadata configuration, and AI assistance can generate common business code with less repetition and more consistency.Sun, 31 May 2026 00:00:00 GMThttps://wzk.icu/en/projects/stop-slop-zh/https://wzk.icu/en/projects/stop-slop-zh/stop-slop-zh is a rule project for Chinese AI writing quality, collecting common AI-flavored phrases, redundant sentence patterns, vague slogans, and mechanical transitions.Sun, 31 May 2026 00:00:00 GMThttps://wzk.icu/en/blog/hive-scd-type2/https://wzk.icu/en/blog/hive-scd-type2/Offline data warehouse needs to save order history state at low cost while supporting daily rollback and change analysis.Sun, 15 Mar 2026 00:00:00 GMThttps://wzk.icu/en/blog/hello-world/https://wzk.icu/en/blog/hello-world/Documenting the complete process of migrating wzk.icu from plain HTML to Astro 6: why I chose Astro, content collection setup, Bun gotchas, and SEO infrastructure building.Fri, 13 Mar 2026 00:00:00 GMThttps://wzk.icu/en/blog/llm-production-engineering/https://wzk.icu/en/blog/llm-production-engineering/Core experience moving LLM applications from prototype to production: context management, error handling, cost control, observability. No basics, just real pitfalls.Tue, 10 Mar 2026 00:00:00 GMThttps://wzk.icu/en/blog/voice-interaction-latency/https://wzk.icu/en/blog/voice-interaction-latency/When building voice interaction systems, latency is the core experience metric.Sun, 08 Mar 2026 00:00:00 GMThttps://wzk.icu/en/blog/k8s-spring-boot-pitfalls/https://wzk.icu/en/blog/k8s-spring-boot-pitfalls/Documenting real issues encountered when migrating Spring Boot microservices to Kubernetes: health checks, ConfigMap hot-reload, JVM memory, graceful shutdown, and log collection.Thu, 05 Mar 2026 00:00:00 GMThttps://wzk.icu/en/projects/wzk-icu/https://wzk.icu/en/projects/wzk-icu/A personal technical portal rebuilt with Astro 6, combining blog publishing, project pages, personal branding, static generation, Markdown content, SEO, and AI-search-friendly structure.Sun, 01 Mar 2026 00:00:00 GMThttps://wzk.icu/en/blog/spring-cloud-gateway-ratelimit/https://wzk.icu/en/blog/spring-cloud-gateway-ratelimit/Recording the complete process of configuring Spring Cloud Gateway rate limiting (Redis token bucket) and circuit breaker (Resilience4j) in production, including pitfalls and tuning experience.Sat, 28 Feb 2026 00:00:00 GMThttps://wzk.icu/en/blog/esp32-home-assistant-sensor/https://wzk.icu/en/blog/esp32-home-assistant-sensor/Build a temperature and humidity sensor using ESP32 + DHT22, integrate with Home Assistant via MQTT, and achieve automation. Complete steps including troubleshooting guide.Fri, 20 Feb 2026 00:00:00 GMThttps://wzk.icu/en/projects/ascendlab/https://wzk.icu/en/projects/ascendlab/AscendLab is an experimental site for browser-side tools, AI workflows, and indie building, exploring how small useful tools and automation flows can become maintainable public works.Thu, 15 Jan 2026 00:00:00 GMThttps://wzk.icu/en/blog/java-217/https://wzk.icu/en/blog/java-217/RocketMQ core terminology and message model quick reference: Producer, Broker, Consumer, Consumer Group, cluster consumption, broadcast consumption, ordered messages, Tag and Queue.Mon, 29 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-218/https://wzk.icu/en/blog/java-218/Complete RocketMQ Java API tutorial, sync/async message producer (Producer), pull consumer (PullConsumer) and push consumer (PushConsumer) code examples.Mon, 29 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-215/https://wzk.icu/en/blog/java-215/Deep dive into RocketMQ consumption modes: Push vs Pull comparison, long polling mechanism, consumption backlog handling and Offset management optimization.Fri, 26 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-216/https://wzk.icu/en/blog/java-216/Complete solution for RocketMQ 4.5.1 startup issues on JDK9+, script modifications (GC parameters, CLASSPATH, java.ext.dirs) and environment configuration.Fri, 26 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-213/https://wzk.icu/en/blog/java-213/RocketMQ (formerly MetaQ) technical evolution history and core architecture deep dive, NameServer, Broker, Producer, Consumer roles and deployment topology.Thu, 25 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-214/https://wzk.icu/en/blog/java-214/Deep dive into RocketMQ core features: ordered consumption, Broker-side filtering, transaction messages, scheduled timing messages, consumption retry.Thu, 25 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-210/https://wzk.icu/en/blog/java-210/Deep dive into RabbitMQ message reliability, Publisher Confirms mechanism, mandatory Return routing failure callback, message persistence and consumer idempotency practice.Wed, 24 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-212/https://wzk.icu/en/blog/java-212/Complete Spring Boot integration with RabbitMQ tutorial, DirectExchange configuration, @RabbitListener message listener and Controller HTTP trigger delivery example.Wed, 24 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-209/https://wzk.icu/en/blog/java-209/Complete Spring AMQP integration with RabbitMQ tutorial, XML configuration to declare queues/exchanges/bindings, RabbitTemplate message sending/receiving and common troubleshooting.Tue, 23 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-211/https://wzk.icu/en/blog/java-211/Spring AMQP JavaConfig annotation configuration to integrate RabbitMQ, ConnectionFactory, RabbitTemplate, Queue Bean configuration and message sending/receiving examples.Tue, 23 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-208/https://wzk.icu/en/blog/java-208/Deep dive into RabbitMQ Topic exchange, * and # wildcard rules, RoutingKey format specifications and Java producer/consumer code examples.Mon, 22 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-206/https://wzk.icu/en/blog/java-206/RabbitMQ fanout broadcast mode Java tutorial, deep dive into Push vs Pull implementation, applicable scenarios and ACK confirmation mechanism.Sat, 20 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-207/https://wzk.icu/en/blog/java-207/RabbitMQ Direct exchange routing mode details, RoutingKey exact match rules, queue multi-binding for log-level consumption and Java code examples.Sat, 20 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-203/https://wzk.icu/en/blog/java-203/Deep dive into RabbitMQ Connection and Channel mechanism, AMQP frame structure packet capture analysis, and common issues and solutions under high concurrency.Fri, 19 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-205/https://wzk.icu/en/blog/java-205/RabbitMQ Work Queue task queue load balancing practice and fanout publish-subscribe mode, including manual ACK, QoS flow control and temporary queue usage guide.Fri, 19 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-202/https://wzk.icu/en/blog/java-202/RabbitMQ production environment RPM installation tutorial, including Erlang compatibility configuration, Docker Compose quick start, common management commands and troubleshooting.Thu, 18 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-204/https://wzk.icu/en/blog/java-204/Complete producer and consumer workflow analysis for RabbitMQ, including Connection/Channel mechanism, default exchange behavior, message acknowledgment mechanism and common troubleshooting.Thu, 18 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-200/https://wzk.icu/en/blog/java-200/RabbitMQ architecture analysis and detailed explanation of four Exchange routing modes: fanout broadcast, direct exact, topic wildcard, headers multi-condition matching.Wed, 17 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-201/https://wzk.icu/en/blog/java-201/Deep dive into RabbitMQ message persistence mechanism: queue index (.idx) + msg_store (.Wed, 17 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-195/https://wzk.icu/en/blog/java-195/Java systems doing async decoupling and event-driven need unified understanding of JMS message model, object model and message types.Tue, 16 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-196/https://wzk.icu/en/blog/java-196/JMS Topic broadcasts in application cluster, causing same business to be repeatedly consumed by multiple nodes.Tue, 16 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-198/https://wzk.icu/en/blog/java-198/E-commerce seckill/ticket-grabbing scenarios with instantaneous traffic peaks, high read/write concurrency. Use pre-static + rate limiting queuingMon, 15 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-197/https://wzk.icu/en/blog/java-197/Implement a minimal runnable message model with queue buffering and async decoupling using Java concurrency.Sun, 14 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-199/https://wzk.icu/en/blog/java-199/Coexisting with traditional IBM MQ, need open source, operatable, scalable, consistency/reliability.Sun, 14 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-190/https://wzk.icu/en/blog/java-190/After distributed service splitting, inter-service communication evolved from synchronous calls to async decoupling, with retry, idempotency, and reliability patterns for message-driven architectures.Fri, 12 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-192/https://wzk.icu/en/blog/java-192/Java service integrating with Memcached needs to understand Spymemcached's thread model, sharding routing and serialization details.Fri, 12 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-193/https://wzk.icu/en/blog/java-193/Java applications wanting to experience Netflix EVCache locally or in small teams, but server not open source, can only self-build based on Memcached.Thu, 11 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-194/https://wzk.icu/en/blog/java-194/Using EVCache in large-scale distributed cache needs understanding of underlying Memcached memory management and eviction behavior.Thu, 11 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-185/https://wzk.icu/en/blog/java-185/Guava Cache is widely used in production, but many lack source-level understanding of LocalCache/Segment/LoadingCache behavior.Wed, 10 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-191/https://wzk.icu/en/blog/java-191/Want to learn Netflix's EVCache system for self-development, but only know it's 'Memcached-based distributed cache'.Wed, 10 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-186/https://wzk.icu/en/blog/java-186/Deep dive into concurrencyLevel, refreshAfterWrite and LoadingCache dynamic loading behavior, with horizontal comparison with custom LinkedHashMap LRU implementation.Tue, 09 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-189/https://wzk.icu/en/blog/java-189/Using Guava Cache for local caching in Java projects, troubleshooting and fixing production issues like OOM, abnormal hit rate, thread blocking and performance regression.Tue, 09 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-187/https://wzk.icu/en/blog/java-187/Guava Cache uses 'lazy cleanup + LRU+FIFO' strategy. Passive deletion and active deletion need to be used together. Deep dive into expireAfterAccess, expireAfterWrite and cleanUp.Mon, 08 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-188/https://wzk.icu/en/blog/java-188/Deep dive into LocalCache, Guava Cache's core implementation class, covering segmented lock design, five queue mechanism, eviction strategy and lazy cleanup principles.Mon, 08 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-184/https://wzk.icu/en/blog/java-184/High concurrency read-heavy business, database can't handle it, need to improve throughput and stability.Wed, 03 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-135-gemini3-pro/https://wzk.icu/en/blog/ai-research-135-gemini3-pro/Explains Gemini 3 Pro's advantages through sparse MoE architecture, million-token context, native multimodal (text/image/video/PDF), thinking depth control (thinking_level), and Deep Think mode.Tue, 02 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-179/https://wzk.icu/en/blog/java-179/For multi-team and cross-account collaboration, OSS needs public access, least privilege, and auditability.Tue, 02 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-134-java-2025/https://wzk.icu/en/blog/ai-research-134-java-2025/In 2025, Java remains the enterprise backend and critical industry workhorse.Mon, 01 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-181/https://wzk.icu/en/blog/java-181/Enterprises and individuals migrating images/audio/video/static resources to Aliyun OSS need stable operations and cost control.Mon, 01 Dec 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-133-java-vs-other/https://wzk.icu/en/blog/ai-research-133-java-vs-other/Comprehensive comparison of Java 21/23 (Virtual Threads/Loom) with Kotlin 2.x, Go 1.22/1.23, Rust 1.7x, Python (PyTorch 2.x/TensorFlow 2.x), Node.js 20/22.Fri, 28 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-182/https://wzk.icu/en/blog/java-182/Two ways for Java applications to integrate with FastDFS: official source compilation (happyfish100/fastdfs-client-java 1.37-SNAPSHOT) and Maven dependency (cn.bestwu / com.github.tobato).Fri, 28 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-132-java-ecosystem-2025/https://wzk.icu/en/blog/ai-research-132-java-ecosystem-2025/Spring Framework 6, Spring Boot 3.x, GraalVM Native Image, Quarkus, Micronaut, and AOT trends shaping the Java ecosystem in 2025.Thu, 27 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-180/https://wzk.icu/en/blog/java-180/Scenario: High concurrency / massive small files (trunk) / single disk or RAID Conclusion: Upgrade to v5.04+, increase max_connections and align with nofileThu, 27 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-131-java-17-21-25/https://wzk.icu/en/blog/ai-research-131-java-17-21-25/Java 17 (2021), Java 21 (2023), Java 25 (2025) language and JVM changes, covering Virtual Threads (Project Loom), Records/Pattern Matching (Project Amber).Wed, 26 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-175/https://wzk.icu/en/blog/java-175/Deploy tracker, storage, and Nginx on three hosts with round-robin upload. Key points: Nginx+mod_fastdfs routing and monitoring verification.Wed, 26 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-130-qwen25-omni/https://wzk.icu/en/blog/ai-research-130-qwen25-omni/Office assistant, education and training, programming and operations, search-enhanced RAG, device control/plugin agents, and companion entertainment.Wed, 19 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-129-qwen25-omni-7b/https://wzk.icu/en/blog/ai-research-129-qwen25-omni-7b/Runs stably at FP16 ~14GB VRAM, with INT8/INT4 quantization (<4GB) enabling deployment on consumer GPUs or edge devices.Tue, 18 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-128-qwen25-omni-training/https://wzk.icu/en/blog/ai-research-128-qwen25-omni-training/Complete training pipeline breakdown for Qwen2.5-Omni: Thinker based on Qwen2.5, vision initialized from Qwen2.5-VL, audio from Whisper-large-v3.Mon, 17 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-127-qwen25-omni-deep/https://wzk.icu/en/blog/ai-research-127-qwen25-omni-deep/Engineering breakdown of Qwen2.5-Omni (2024-2025) Thinker-Talker dual-core architecture: unified Transformer decoder for text/image/video/audio fusion, TMRoPE.Sun, 16 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-126-tesla-fsd-v14/https://wzk.icu/en/blog/ai-research-126-tesla-fsd-v14/Tesla developed AI-driven high-precision occupancy network, converting multi-camera video into continuous 3D Signed Distance Field (SDF).Wed, 12 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-125-tesla-fsd-business/https://wzk.icu/en/blog/ai-research-125-tesla-fsd-business/As of end 2022, Tesla had ~$2.9 billion in FSD-related deferred revenue Q4 2022 recognized $324 million in FSD revenueTue, 11 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-124-tesla-fsd-v14-analysis/https://wzk.icu/en/blog/ai-research-124-tesla-fsd-v14-analysis/Tesla FSD V14 real-world performance and road tests, comparing V13.2 on urban roads and highways: key disengagement metrics, lane changes/ramps, destination arrival, and long-tail scenarios.Mon, 10 Nov 2025 00:00:00 GMThttps://wzk.icu/en/projects/teslamate/https://wzk.icu/en/projects/teslamate/A Tesla vehicle data dashboard built on TeslaMate, Grafana, InfluxDB, and PostgreSQL, with trip records, charging analysis, energy metrics, alerts, and weekly reports.Mon, 10 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-123-fsd-v14-vs-v12/https://wzk.icu/en/blog/ai-research-123-fsd-v14-vs-v12/3D environment reconstruction Precision: 10cm (3× improvement over V12's ~33cm resolution) Multi-frame spatiotemporal fusion for dynamic object trackingFri, 07 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-121-deepseek-ocr/https://wzk.icu/en/blog/ai-research-121-deepseek-ocr/Frontier approaches and engineering implementation for DeepSeek-OCR (2025, including 3B parameter direction).Wed, 05 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-119-deepseek-ocr-deployment/https://wzk.icu/en/blog/ai-research-119-deepseek-ocr-deployment/Comprehensive guide for DeepSeek-OCR local/private deployment based on Python 3.12, PyTorch 2.6.0, Transformers 4.46.3 and FlashAttention 2.7.3.Tue, 04 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-120-deepseek-ocr-start/https://wzk.icu/en/blog/ai-research-120-deepseek-ocr-start/Complete getting started path and engineering essentials for DeepSeek-OCR (as of 2025), covering environment setup (Python/PyTorch 2.x, Transformers 4.Tue, 04 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-118-mobile-aloha/https://wzk.icu/en/blog/ai-research-118-mobile-aloha/Mobile-ALOHA: An open-source mobile manipulation solution combining mobile chassis and dual-arm collaboration.Mon, 03 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-117-tesla-fsd-vision/https://wzk.icu/en/blog/ai-research-117-tesla-fsd-vision/Detailed analysis of Tesla's 3D rendering (Occupancy Network): multi-camera spatiotemporal fusion → voxel occupancy → bird's eye coordinates.Sat, 01 Nov 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-research-116-tesla-hw3-hw4/https://wzk.icu/en/blog/ai-research-116-tesla-hw3-hw4/Comprehensive comparison of Tesla HW3.0 and HW4.0 hardware: camera resolution upgraded from 1.2MP to 5MP with better HDR/night visionFri, 31 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-108-robot-training/https://wzk.icu/en/blog/ai-investigation-108-robot-training/Full robot training pipeline: pre-training, fine-tuning (LoRA), reinforcement learning, imitation learning, and human feedback for safe autonomous decision-making.Mon, 20 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-107-rl-data-format/https://wzk.icu/en/blog/ai-investigation-107-rl-data-format/Constructed in state-action-reward sequence form, supporting spatiotemporal understanding of models like Transformers.Sat, 18 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-106-robot-data-collection/https://wzk.icu/en/blog/ai-investigation-106-robot-data-collection/Core data collection methods and application scenarios, covering over ten methods from manual entry, sensor collection, web crawlers, API calls, log collection.Fri, 17 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-105-robot-learning-data/https://wzk.icu/en/blog/ai-investigation-105-robot-learning-data/Data collection is a critical step in robot learning development, covering demonstration video collection, trajectory recording, state-action pair generation, and data quality control strategies.Thu, 16 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-104-robot-deployment/https://wzk.icu/en/blog/ai-investigation-104-robot-deployment/AI model deployment optimization guide: ONNX conversion, TensorRT/OpenVINO inference engines, quantization (FP16/INT8), and real-time robotics applications.Wed, 15 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-103-embodied-ai/https://wzk.icu/en/blog/ai-investigation-103-embodied-ai/Comprehensive overview of embodied AI tech stack: hardware (GPU, sensors, actuators), software (ROS, simulation), and algorithms (deep learning, RL, VLA models).Tue, 14 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-102-robot-ml-trends/https://wzk.icu/en/blog/ai-investigation-102-robot-ml-trends/Different types of robots have huge differences in structure, tasks and control methods, so AI algorithm adaptation strategies also need to be tailored.Mon, 13 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-101-vla-rt/https://wzk.icu/en/blog/ai-investigation-101-vla-rt/Modern AI robot control methods are undergoing a major transition from reinforcement learning and imitation learning to multimodal agents driven by large models.Sun, 12 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-100-rl-il-transformer/https://wzk.icu/en/blog/ai-investigation-100-rl-il-transformer/Modern AI methods for robot control cover Reinforcement Learning (RL), Imitation Learning (IL), and Transformer-based large model methods.Sat, 11 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-99-sensor-fusion/https://wzk.icu/en/blog/ai-investigation-99-sensor-fusion/Sensor Fusion is a core technology in autonomous driving, robotics and smart security.Fri, 10 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-98-visual-slam/https://wzk.icu/en/blog/ai-investigation-98-visual-slam/Visual SLAM is a technology that achieves autonomous positioning and environment mapping without relying on LiDAR, using only cameras.Thu, 09 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-97-slam-algorithms/https://wzk.icu/en/blog/ai-investigation-97-slam-algorithms/Multi-sensor fusion and SLAM are core technologies for robot perception and navigation.Wed, 08 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-96-robot-testing/https://wzk.icu/en/blog/ai-investigation-96-robot-testing/Complete guide to robot scenario testing, covering three dimensions: environment testing, load testing, and anomaly testing.Tue, 07 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-95-robot-testing-2/https://wzk.icu/en/blog/ai-investigation-95-robot-testing-2/Camera Instant Frame Loss: 5-100ms frame drop LiDAR Noise Surge: Random noise 5-20% IMU Data Jump: 1-3x normal valuesSun, 05 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-94-robot-validation/https://wzk.icu/en/blog/ai-investigation-94-robot-validation/This article explores three core aspects of robot algorithm real machine validation - test platform selection, deployment process design and interface integration.Sat, 04 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-93-simulation-tools/https://wzk.icu/en/blog/ai-investigation-93-simulation-tools/Simulation tools are an important part of robot R&D, enabling algorithm verification and system debugging in risk-free environments, accelerating iteration.Fri, 03 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-92-motion-control/https://wzk.icu/en/blog/ai-investigation-92-motion-control/Robot motion control can be divided into two categories: traditional model-based methods and deep learning-based intelligent control.Thu, 02 Oct 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-91-data-annotation/https://wzk.icu/en/blog/ai-investigation-91-data-annotation/In robot vision and perception model training, high-quality multi-modal data annotation tools are crucial.Tue, 30 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-90-robot-middleware/https://wzk.icu/en/blog/ai-investigation-90-robot-middleware/Modern robot systems require efficient data collection and communication middleware to connect sensors, controllers and computing units, achieving collaborative perception.Mon, 29 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-89-autonomic-nervous/https://wzk.icu/en/blog/ai-investigation-89-autonomic-nervous/Individual differences in autonomic nervous system (ANS) profoundly affect disease susceptibility.Sat, 27 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-88-hrv/https://wzk.icu/en/blog/ai-investigation-88-hrv/Higher HRV: Stronger vagal tone, more relaxed Lower HRV: Chronic stress, fatigue, cardiovascular diseaseFri, 26 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-87-hrv-metrics/https://wzk.icu/en/blog/ai-investigation-87-hrv-metrics/HRV calculation methods mainly include time domain and frequency domain.Thu, 25 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-85-autonomic-assessment/https://wzk.icu/en/blog/ai-investigation-85-autonomic-assessment/Measurement technology: ECG, PPG Analysis: Time domain, frequency domain, nonlinearTue, 23 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-84-fat-loss-maintenance/https://wzk.icu/en/blog/ai-investigation-84-fat-loss-maintenance/Maintaining low body fat is more challenging than losing it. The human body is naturally inclined to store energy.Mon, 22 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-83-diet-guide/https://wzk.icu/en/blog/ai-investigation-83-diet-guide/During fat loss, intake is recommended at 1900-2100 kcal, with daily deficit of 300-500 kcal.Sat, 20 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-82-skinny-fitness/https://wzk.icu/en/blog/ai-investigation-82-skinny-fitness/For lean people, the key to fat loss and body shaping is 'muscle building primary, cardio auxiliary'.Fri, 19 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-81-body-fat/https://wzk.icu/en/blog/ai-investigation-81-body-fat/Body fat percentage affects both appearance and health directly. Healthy body fat for men is 10%~20%, ideal around 15%; for women it's between 18%~28%.Thu, 18 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-80-fat-loss-science/https://wzk.icu/en/blog/ai-investigation-80-fat-loss-science/The core mechanism of fat loss is energy deficit: when calorie intake is less than consumption, the body uses fat stores for energy.Wed, 17 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-78-lfp-battery/https://wzk.icu/en/blog/ai-investigation-78-lfp-battery/Shallow charge frequent and deep discharge charge have significantly different impacts on LFP battery life.Mon, 15 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-79-lfp-battery-decay/https://wzk.icu/en/blog/ai-investigation-79-lfp-battery-decay/The widespread use of LFP batteries in Tesla and other models has accumulated large amounts of real-world lifetime data.Sun, 14 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-77-lfp-vs-nmc/https://wzk.icu/en/blog/ai-investigation-77-lfp-vs-nmc/Lithium Iron Phosphate (LFP) batteries, with their stable olivine crystal structure, significantly outperform ternary lithium (NMC/NCA) batteries in cycle life and safety.Sat, 13 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-76-embodied-ai-society-impact/https://wzk.icu/en/blog/ai-investigation-76-embodied-ai-society-impact/The widespread application of embodied AI is profoundly changing social structure.Fri, 12 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-75-llm-to-lbm/https://wzk.icu/en/blog/ai-investigation-75-llm-to-lbm/The integration of Large Language Models (LLM) with robot real-time control is driving intelligent upgrades in robotics.Thu, 11 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-74-meta-learning-sim2real/https://wzk.icu/en/blog/ai-investigation-74-meta-learning-sim2real/This article explores fast learning capabilities of embodied AI agents, focusing on meta-learning and few-shot learning methods.Wed, 10 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-73-embodied-ai-future/https://wzk.icu/en/blog/ai-investigation-73-embodied-ai-future/In the next decade, embodied AI will undergo paradigm shifts: centered on 'pre-trained world models + online learning', software-hardware collaboration and interdisciplinary fusion accelerating.Tue, 09 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-72-embodied-ai-challenges/https://wzk.icu/en/blog/ai-investigation-72-embodied-ai-challenges/Embodied AI development faces six core challenges: data scarcity, hardware limitations, training efficiency, cost bottlenecks, standardization and industrial ecosystem, safety and ethics.Mon, 08 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-71-ros-tesla-optimus/https://wzk.icu/en/blog/ai-investigation-71-ros-tesla-optimus/Typical practices of embodied AI in architecture, capabilities and applications.Sun, 07 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-70-embodied-ai-industry/https://wzk.icu/en/blog/ai-investigation-70-embodied-ai-industry/Embodied AI is leading a new round of technological revolution. The market size is expected to grow from $2.53 billion in 2024 to $8.76 billion in 2033, with a CAGR of 15%.Sat, 06 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-69-embodied-ai-capabilities/https://wzk.icu/en/blog/ai-investigation-69-embodied-ai-capabilities/Systematic overview of five key capability dimensions of embodied AI: intelligent algorithms, high-performance hardware, simulation and virtual environments, embedded and software systems.Thu, 04 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-68-embodied-ai-applications/https://wzk.icu/en/blog/ai-investigation-68-embodied-ai-applications/Systematically explore the application prospects and development trends of embodied AI in multiple fields, covering core scenarios including home, industry, healthcare.Wed, 03 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-67-embodied-ai-core-tech/https://wzk.icu/en/blog/ai-investigation-67-embodied-ai-core-tech/Embodied AI technology can be summarized as a 'perception-decision-control-learning-interaction' closed-loop system.Tue, 02 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-66-robot-arm-software/https://wzk.icu/en/blog/ai-investigation-66-robot-arm-software/Robotic arm software algorithm system covers core technologies including kinematics, trajectory planning, robot vision, artificial intelligence and machine learning.Mon, 01 Sep 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-65-robot-control-plc-mpc/https://wzk.icu/en/blog/ai-investigation-65-robot-control-plc-mpc/Implementation of robotic arm systems requires software-hardware collaboration, with strict requirements from control layer to safety mechanisms. Control systems have evolved from PLC to MPC.Sun, 31 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-64-robot-arm-components/https://wzk.icu/en/blog/ai-investigation-64-robot-arm-components/Building and controlling a high-performance robotic arm requires integrating multiple core modules including motor drive, reducer, mechanical structure, sensor system, controller and end effector.Sat, 30 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-63-robot-arm-applications/https://wzk.icu/en/blog/ai-investigation-63-robot-arm-applications/Modern robotic arms undertake various key tasks in industrial automation: handling, assembly, welding, spraying, grinding, unstructured grasping, machining, inspection and testing.Fri, 29 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-62-robot-arm-scenarios/https://wzk.icu/en/blog/ai-investigation-62-robot-arm-scenarios/Robotic arms, as important tools for modern automation, have been widely applied in manufacturing, healthcare, agriculture, services and scientific research.Thu, 28 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-106/https://wzk.icu/en/blog/java-106/Database sharding explained: vertical partitioning, horizontal partitioning, vertical sharding, vertical table splitting, horizontal table splitting.Thu, 28 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-61-robot-types/https://wzk.icu/en/blog/ai-investigation-61-robot-types/Complete industrial robot spectrum: articulated robotic arms suitable for complex trajectory tasks like automotive manufacturingWed, 27 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-108/https://wzk.icu/en/blog/java-108/MHA high availability solution: Manager, Node, fault handling, failover.Wed, 27 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-60-robot-arm-trends/https://wzk.icu/en/blog/ai-investigation-60-robot-arm-trends/Robotic arm technology development shows five major trends: high precision, collaboration, lightweight, intelligence, and connectivity.Tue, 26 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-86-autonomic-balance/https://wzk.icu/en/blog/ai-investigation-86-autonomic-balance/Systematically expounds the balance mechanism of sympathetic and parasympathetic nerves in the human autonomic nervous system, deeply analyzing their physiological effects.Tue, 26 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-110/https://wzk.icu/en/blog/java-110/Read-write separation principles, implementation methods, master-slave lag solutions, database middleware.Mon, 25 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-59-robot-career/https://wzk.icu/en/blog/ai-investigation-59-robot-career/The robotics industry is experiencing rapid development, with surging demand for interdisciplinary talent with mechanical, electronic, control and software capabilities.Sun, 24 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-58-robot-applications/https://wzk.icu/en/blog/ai-investigation-58-robot-applications/As a general-purpose technology, robots are widely used in manufacturing, healthcare, agriculture, logistics and home sectors.Sat, 23 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-57-robot-categories/https://wzk.icu/en/blog/ai-investigation-57-robot-categories/Robots can be divided into five major categories based on function and form: industrial robots, service robots, humanoid robots, mobile robots, and specialty robots.Fri, 22 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-56-robot-evolution/https://wzk.icu/en/blog/ai-investigation-56-robot-evolution/Robotics has undergone profound evolution from early hydraulic drive and analog control to modern electric drive, digital control and perception systems.Thu, 21 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-55-robot-history/https://wzk.icu/en/blog/ai-investigation-55-robot-history/Since the term 'Robot' was first introduced in 1921, robotics has undergone a century of evolution from science fiction to reality.Wed, 20 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-54-bigdata-industry/https://wzk.icu/en/blog/ai-investigation-54-bigdata-industry/Big data has achieved deep integration in finance, e-commerce, internet, communications, manufacturing, healthcare, education and other industries, becoming the core engine for business innovation.Mon, 18 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-53-bigdata-talent/https://wzk.icu/en/blog/ai-investigation-53-bigdata-talent/The talent structure in the big data industry shows characteristics of youth and rapid growth. The 25-30 age group is the main force, while 30-35 year-olds are gradually becoming the core strength.Sun, 17 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-52-bigdata-tech-trends/https://wzk.icu/en/blog/ai-investigation-52-bigdata-tech-trends/Big data technology is undergoing a new wave of transformation. Lakehouse architecture combines the advantages of data lakes and data warehouses.Sat, 16 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-97/https://wzk.icu/en/blog/java-97/Concurrent transaction problems: lost update, dirty read, non-repeatable read, phantom read, lock mechanism: shared lock, exclusive lock, MVCC.Fri, 15 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-51-bigdata-evolution/https://wzk.icu/en/blog/ai-investigation-51-bigdata-evolution/Big data technology evolution: MapReduce replaced by Spark, Storm replaced by Flink, Pig/Hive gradually phased out.Thu, 14 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-100/https://wzk.icu/en/blog/java-100/ACID properties: atomicity, consistency, isolation, durability, WAL mechanism.Thu, 14 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-50-bigdata-evolution/https://wzk.icu/en/blog/ai-investigation-50-bigdata-evolution/Two decades of big data evolution: from 2006 MapReduce batch processing to 2013 Spark in-memory computing, to 2019 Flink real-time computing.Wed, 13 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-98/https://wzk.icu/en/blog/java-98/Slow query log, mysqldumpslow, EXPLAIN analysis, common optimization solutions.Wed, 13 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-49-bigdata-history/https://wzk.icu/en/blog/ai-investigation-49-bigdata-history/Big data development began in 1997 when NASA proposed the concept, 2003-2006 Google published GFS, MapReduce, Bigtable three major papers leading distributed computing revolution.Tue, 12 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-99/https://wzk.icu/en/blog/java-99/filesort sorting, index sorting, two-pass sorting, single-pass sorting, clustered index and secondary index sorting.Tue, 12 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-91/https://wzk.icu/en/blog/java-91/Table lookup queries, covering index, leftmost prefix principle, LIKE queries, NULL value handling.Mon, 11 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-94/https://wzk.icu/en/blog/java-94/EXPLAIN command: selecttype, type, possiblekeys, key, rows, key_len, Extra.Fri, 08 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-93/https://wzk.icu/en/blog/java-93/Clustered index (secondary index, primary key index), secondary index, table lookup query, covering index.Thu, 07 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-95/https://wzk.icu/en/blog/java-95/Index principles: B+Tree, Hash index, binary search, InnoDB adaptive hash index.Wed, 06 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-92/https://wzk.icu/en/blog/java-92/Index types: BTree, Hash, FULLTEXT, RTree, regular index, unique index, primary key index, composite index, clustered index.Tue, 05 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-90/https://wzk.icu/en/blog/java-90/Binlog three recording modes: STATEMENT, ROW, MIXED, file format, write mechanism, master-slave replication.Mon, 04 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-87/https://wzk.icu/en/blog/java-87/Undo Log for rollback, Redo Log for redo, transaction atomicity, durability, MVCC.Sat, 02 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-01-mindfulness/https://wzk.icu/en/blog/ai-investigation-01-mindfulness/Mindfulness meditation is a scientifically validated mind-body regulation method.Fri, 01 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-86/https://wzk.icu/en/blog/java-86/InnoDB storage structure: Tablespace, Segment, Extent, Page, Row. Row formats: REDUNDANT, COMPACT, DYNAMIC, COMPRESSEDFri, 01 Aug 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-88/https://wzk.icu/en/blog/java-88/MySQL 5.7/8.0 storage engine optimization, InnoDB thread model: IO Thread, Purge Thread, Page Cleaner Thread, Master ThreadThu, 31 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-48-tcm-western-medicine/https://wzk.icu/en/blog/ai-investigation-48-tcm-western-medicine/Compare differences and advantages of Chinese and Western medicine in treating common diseases and chronic diseases.Wed, 30 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-89/https://wzk.icu/en/blog/java-89/InnoDB disk structure: system tablespace, file-per-table tablespace, general tablespace, temporary tablespace, Doublewrite Buffer, Redo Log, Undo Log - complete storage architecture guide.Wed, 30 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-47-nutrition-strategy/https://wzk.icu/en/blog/ai-investigation-47-nutrition-strategy/Extra nutrient supplementation should be based on individual assessment, not blind following.Tue, 29 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-81/https://wzk.icu/en/blog/java-81/InnoDB memory structure: Buffer Pool, Change Buffer, Adaptive Hash Index, Log BufferTue, 29 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-46-supplements-chronic-disease/https://wzk.icu/en/blog/ai-investigation-46-supplements-chronic-disease/Multiple large-scale clinical studies show that for ordinary healthy people, long-term intake of multivitamin supplements has limited overall effect in preventing cardiovascular disease, cancer.Mon, 28 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-82/https://wzk.icu/en/blog/java-82/MySQL storage engine comparison: InnoDB, MyISAM, Memory, Archive - transactions, locking mechanisms, index structures.Mon, 28 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-45-nutrition-gender/https://wzk.icu/en/blog/ai-investigation-45-nutrition-gender/Explore main differences in nutritional supplementation between men and women, focusing on different needs for key nutrients like iron, calcium, and folate.Sun, 27 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-44-supplements-fishoil-calcium/https://wzk.icu/en/blog/ai-investigation-44-supplements-fishoil-calcium/Survey research on whether nutritional supplements like fish oil, calcium tablets, and vitamin C need extra intake. Analyze applicable scenarios and precautions for each nutritional supplement.Sat, 26 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-84/https://wzk.icu/en/blog/java-84/MySQL connection mechanism, half-duplex communication, query cache, query optimizer explained.Sat, 26 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-43-multivitamin-young-old/https://wzk.icu/en/blog/ai-investigation-43-multivitamin-young-old/Multivitamin supplements can fill nutritional gaps caused by unbalanced diet, suitable for elderly, vegetarians, pregnant women and other specific groups.Fri, 25 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-83/https://wzk.icu/en/blog/java-83/MySQL four-layer architecture: network connection layer, service layer, storage engine layer, system file layer.Fri, 25 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-42-quantization-summary/https://wzk.icu/en/blog/ai-investigation-42-quantization-summary/Survey outline for multimodal large model quantization schemes: from FP32 to INT4. Core goal is model capability retention, compression efficiency 50-75%, inference speedup 2-4x.Thu, 24 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-85/https://wzk.icu/en/blog/java-85/MySQL development history, main branches, architecture evolution (standalone -> master-slave -> sharding -> cloud database).Thu, 24 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-41-qwen25-vl/https://wzk.icu/en/blog/ai-investigation-41-qwen25-vl/Qwen2.5-VL is the new generation multimodal large model launched by Alibaba, significantly leading in visual understanding, video analysis, and cross-modal reasoning.Wed, 23 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-40-multimodal-models/https://wzk.icu/en/blog/ai-investigation-40-multimodal-models/Multimodal large models are developing rapidly, with representative models like BLIP-2, MiniGPT-4, Flamingo, LLaVA, and Qwen2.5-VL emerging.Tue, 22 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-39-finetune-quantization/https://wzk.icu/en/blog/ai-investigation-39-finetune-quantization/In multimodal large model optimization, the order choice of fine-tuning and quantization directly affects the final model's performance and efficiency.Mon, 21 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-38-vlm-evaluation/https://wzk.icu/en/blog/ai-investigation-38-vlm-evaluation/To systematically evaluate the impact of model quantization on performance, need to combine multiple vision-language datasets and metrics.Sun, 20 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-37-multimodal-impact/https://wzk.icu/en/blog/ai-investigation-37-multimodal-impact/Model quantization compresses FP32 weights into low-precision representations, significantly reducing inference resource consumption.Sat, 19 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-36-multimodal-quantization/https://wzk.icu/en/blog/ai-investigation-36-multimodal-quantization/This comprehensive overview systematically introduces mainstream quantization techniques in multimodal models.Fri, 18 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-35-coffee-taste/https://wzk.icu/en/blog/ai-investigation-35-coffee-taste/The taste experience differences between homemade coffee and chain budget coffee stem from multiple factors including ingredient quality, production process, and psychological feelings.Thu, 17 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-34-coffee-beverage-trend/https://wzk.icu/en/blog/ai-investigation-34-coffee-beverage-trend/China's chain coffee market is experiencing beverage-ization change—coffee products increasingly becoming milk-tea-like.Wed, 16 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-33-coffee-strategy/https://wzk.icu/en/blog/ai-investigation-33-coffee-strategy/Brands like Luckin and Cotti use subsidies to gain market share, driving user growth, but also face risks of cost inversion and profit pressure.Tue, 15 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-32-coffee-price-war/https://wzk.icu/en/blog/ai-investigation-32-coffee-price-war/China's brewed coffee market is shifting from space experience to price+efficiency driven, with local brands like Luckin, Cotti, and Lucky Cup accelerating penetration.Mon, 14 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-31-memory-strategies/https://wzk.icu/en/blog/ai-investigation-31-memory-strategies/8 evidence-based strategies for programmers to combat forgetting: Ebbinghaus forgetting curve can be quantified.Sun, 13 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-30-programmer-memory/https://wzk.icu/en/blog/ai-investigation-30-programmer-memory/Programmers often experience 'forgetting after learning'. Effective strategies include: note organization, spaced repetition (Anki), active recall, project practice, and output sharing.Sat, 12 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-29-perfect-partner/https://wzk.icu/en/blog/ai-investigation-29-perfect-partner/Dating focuses on emotional attraction, marriage values responsibility, communication, and values more.Fri, 11 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-28-china-us-dating/https://wzk.icu/en/blog/ai-investigation-28-china-us-dating/Dating perspective differences between China and US: first love age is approximately 17-18 in China, 15-17 in US. Number of relationships before marriage: 2-3 in China, 4-6 in US.Thu, 10 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-27-social-media-reallocation/https://wzk.icu/en/blog/ai-investigation-27-social-media-reallocation/Heavy social media users have higher rates of depression, anxiety, and loneliness, plus more sleep problems.Wed, 09 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-26-social-media-usage/https://wzk.icu/en/blog/ai-investigation-26-social-media-usage/Global netizens spend an average of 2 hours and 21 minutes on social media daily. Chinese users spend 1.55 hours on Douyin, with short video usage peaking at midnight.Tue, 08 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-25-savings-vs-spending/https://wzk.icu/en/blog/ai-investigation-25-savings-vs-spending/Should young people save more or spend more? Experts recommend 'save first, then spend', following the 50/30/20 rule: 50% needs, 30% wants, 20% savings. Build 3-6 months emergency fund.Mon, 07 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-24-youth-spending/https://wzk.icu/en/blog/ai-investigation-24-youth-spending/Comparison of savings levels, consumption structure, and financial concepts among young people in China, the US, and Japan.Sun, 06 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-23-time-management-metrics/https://wzk.icu/en/blog/ai-investigation-23-time-management-metrics/Time management can significantly improve efficiency and sense of achievement.Sat, 05 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-22-time-management-tools/https://wzk.icu/en/blog/ai-investigation-22-time-management-tools/More than 80% of people do not have a formal time management system. This note compares planning habits, Apple ecosystem tools, and a practical workflow for building a repeatable personal planning system.Fri, 04 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-21-time-management/https://wzk.icu/en/blog/ai-investigation-21-time-management/Effective time management can improve efficiency, reduce stress, and improve work-life balance. Scientific evidence includes psychology, behavioral science, cognitive science, and more.Thu, 03 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-20-mindfulness-health/https://wzk.icu/en/blog/ai-investigation-20-mindfulness-health/Mindfulness meditation is a scientifically validated mind-body practice.Wed, 02 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-19-pm-conflict-advice/https://wzk.icu/en/blog/ai-investigation-19-pm-conflict-advice/Reducing conflicts between programmers and product managers requires: improving requirements process, enhancing communication skills, aligning goals, agile practices, managing expectations.Tue, 01 Jul 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-18-pm-conflict-solution/https://wzk.icu/en/blog/ai-investigation-18-pm-conflict-solution/Resolving conflicts between programmers and product managers requires: establishing clear communication mechanisms, improving document quality, fairly negotiating priorities.Mon, 30 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-17-pm-conflict-factors/https://wzk.icu/en/blog/ai-investigation-17-pm-conflict-factors/Conflicts between programmers and product managers show a 'bell curve' distribution. Influencing factors include project scale, team experience, company type, work mode, and management systems.Sun, 29 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-16-pm-conflict/https://wzk.icu/en/blog/ai-investigation-16-pm-conflict/Conflicts between programmers and product managers most frequently occur in four key stages: requirements analysis, development implementation, schedule planning, and testing acceptance.Sat, 28 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-51/https://wzk.icu/en/blog/java-51/In-depth analysis of distributed services covering distributed consistency including strong consistency, weak consistency, monotonic read consistency, and eventual consistency models.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-56/https://wzk.icu/en/blog/java-56/This article provides an in-depth analysis of the Three-Phase Commit protocol (3PC) and its differences from Two-Phase Commit (2PC) in distributed transaction processing.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-57/https://wzk.icu/en/blog/java-57/This article provides an in-depth analysis of the Two-Phase Commit protocol (2PC) for distributed transactions, covering ACID properties and practical implementation.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-58/https://wzk.icu/en/blog/java-58/This article provides an in-depth analysis of the Paxos distributed consensus algorithm, covering algorithm optimization and how to guarantee liveness properties.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-59/https://wzk.icu/en/blog/java-59/This article provides an in-depth analysis of the CAP theorem in distributed systems, explaining Consistency, Availability, and Partition Tolerance trade-offs.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-60/https://wzk.icu/en/blog/java-60/This article provides an in-depth analysis of the Paxos distributed consensus algorithm, explaining Proposal and Acceptor roles with detailed derivation process.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-61/https://wzk.icu/en/blog/java-61/This article provides an in-depth analysis of RPC and RMI network communication in distributed systems, with practical Java code implementations.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-62/https://wzk.icu/en/blog/java-62/This article provides a detailed analysis of BIO, NIO, and AIO network I/O models in distributed systems, with practical Java code examples.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-63/https://wzk.icu/en/blog/java-63/This article provides an in-depth analysis of heartbeat detection mechanisms and high availability design principles in distributed systems.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-64/https://wzk.icu/en/blog/java-64/This article provides a comprehensive explanation of the Raft distributed consensus algorithm, covering leader election, log replication, and safety mechanisms.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-65/https://wzk.icu/en/blog/java-65/This article provides an in-depth analysis of RPC and RMI communication technologies in distributed systems, covering principles, architecture, and implementations.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-66/https://wzk.icu/en/blog/java-66/This article provides a detailed guide to implementing a custom RPC framework using Netty, with complete code examples for provider and consumer.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-67/https://wzk.icu/en/blog/java-67/This article provides a comprehensive introduction to Apache Dubbo RPC framework, covering core features, service governance, and getting started guide.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-68/https://wzk.icu/en/blog/java-68/This article provides a practical guide to implementing Netty-based server and client applications with detailed Java code examples.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-69/https://wzk.icu/en/blog/java-69/This article provides an in-depth analysis of Netty's EventLoop mechanism, thread model, and core components for building high-performance network applications.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-70/https://wzk.icu/en/blog/java-70/This article provides an in-depth analysis of software architecture evolution from monolithic to distributed to microservices architectures, covering RPC fundamentals.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-71/https://wzk.icu/en/blog/java-71/This article provides a comprehensive guide to Dubbo Admin visual management platform, covering installation methods, source code compilation, and Docker deployment.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-72/https://wzk.icu/en/blog/java-72/This article provides a detailed guide to the Dubbo consumer module, covering configuration to proxy generation, with practical code examples.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-73/https://wzk.icu/en/blog/java-73/This article provides a detailed explanation of the Dubbo producer module, covering key components, startup flow, and production environment best practices.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-74/https://wzk.icu/en/blog/java-74/This article provides an in-depth analysis of Java SPI mechanism, explaining the evolution from JDK SPI to Dubbo's plugin-based extension system.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-75/https://wzk.icu/en/blog/java-75/This article provides a guide to setting up Dubbo parent project configuration with detailed POM and code examples for multi-module Maven projects.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-76/https://wzk.icu/en/blog/java-76/This article provides a comprehensive analysis of Dubbo's Adaptive and Filter interceptor mechanisms for dynamic extension and service chain enhancement.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-77/https://wzk.icu/en/blog/java-77/Comprehensive analysis of Dubbo load balancing strategies, configuration methods, and custom implementation guide covering Random, RoundRobin, LeastActive, and ConsistentHash algorithms.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-78/https://wzk.icu/en/blog/java-78/This article provides an in-depth analysis of Dubbo dynamic service degradation, from avalanche prevention to second-level configuration center activation.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-79/https://wzk.icu/en/blog/java-79/This article provides a comprehensive analysis of Dubbo load balancing strategies and asynchronous call implementation with practical examples.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-80/https://wzk.icu/en/blog/java-80/This article provides an in-depth analysis of Dubbo's dynamic routing architecture, from rule design to production system integration.Wed, 25 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-13-llm-agent/https://wzk.icu/en/blog/ai-investigation-13-llm-agent/2024 is called the 'Year of Agents'. LLM trends show parallel development of 'bigger and stronger' and 'smaller and more specialized'.Tue, 24 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-52/https://wzk.icu/en/blog/java-52/In-depth guide to distributed services covering basic concepts, cluster comparison, common patterns, communication methods, and detailed explanation of three states.Tue, 24 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-55/https://wzk.icu/en/blog/java-55/In-depth guide to Tomcat performance optimization covering JVM memory model, garbage collection GC, and Tomcat configuration optimization with practical examples.Mon, 23 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-15-product-building/https://wzk.icu/en/blog/ai-investigation-15-product-building/Building products from 0 to 1 requires mastering methodologies like Lean Startup and Design Thinking, and validating directions through requirement verification, user interviews.Sat, 21 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-54/https://wzk.icu/en/blog/java-54/In-depth guide to Tomcat covering SSL working principle, performance optimization, parameter configuration, and JVM optimization with practical examples.Sat, 21 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-14-report-writing/https://wzk.icu/en/blog/ai-investigation-14-report-writing/How to write excellent research reports, slide presentations, and technical sharing sessions?Fri, 20 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-53/https://wzk.icu/en/blog/java-53/In-depth analysis of Tomcat class loading mechanism covering parent delegation model, lifecycle, and plugin architecture with detailed explanations.Fri, 20 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-48/https://wzk.icu/en/blog/java-48/Step-by-step guide to implementing MiniCat part 3 covering startup testing, StaticResourceUtil, and Servlet implementation with complete code examples.Thu, 19 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-12-llm-applications/https://wzk.icu/en/blog/ai-investigation-12-llm-applications/Major LLM application directions in 2024-2025 include enterprise applications (code assistance, customer service, knowledge management) and consumer applications.Wed, 18 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-49/https://wzk.icu/en/blog/java-49/Step-by-step guide to implementing MiniCat part 2 covering HttpServlet, Request, RequestProcessor components with detailed implementation code.Wed, 18 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-11-running-nutrition/https://wzk.icu/en/blog/ai-investigation-11-running-nutrition/Running nutrition strategies should focus on carbohydrates (50-65% of daily energy), protein (1.2-1.6g/kg), and fat (20-30%). Consume 15-25g of protein within 30 minutes after training.Tue, 17 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-47/https://wzk.icu/en/blog/java-47/Step-by-step guide to implementing a Mini Tomcat called MiniCat covering Bootstrap, Socket server, Request/Response handling with practical code examples.Tue, 17 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-09-running-tips/https://wzk.icu/en/blog/ai-investigation-09-running-tips/The optimal way to run is to keep heart rate between 60% and 80% of maximum heart rate. Post-run nutrition should include carbohydrates and protein promptly.Mon, 16 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-46/https://wzk.icu/en/blog/java-46/In-depth analysis of Tomcat core architecture covering detailed configuration of main tags including Server, Service, Executor, Engine with practical examples.Mon, 16 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-07-running-distance/https://wzk.icu/en/blog/ai-investigation-07-running-distance/Running has significant benefits for cardiovascular health, weight management, and psychological state.Sun, 15 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-06-cold-shower-guide/https://wzk.icu/en/blog/ai-investigation-06-cold-shower-guide/As a health intervention, cold showers can provide benefits such as refreshing, immune enhancement, and recovery promotion when practiced scientifically.Sat, 14 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-50/https://wzk.icu/en/blog/java-50/In-depth analysis of Tomcat core architecture covering Catalina container full analysis, startup flow, and threading mechanism with comprehensive explanations.Sat, 14 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-05-cold-hot-shower/https://wzk.icu/en/blog/ai-investigation-05-cold-hot-shower/Cold showers have positive effects on mental health, including improved alertness, stress and depression relief, and enhanced psychological resilience.Fri, 13 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-43/https://wzk.icu/en/blog/java-43/In-depth analysis of Tomcat core architecture and processing flow covering Coyote IO model and protocol with comprehensive explanations.Fri, 13 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-04-cold-shower/https://wzk.icu/en/blog/ai-investigation-04-cold-shower/Cold showers typically refer to showering with water temperature at or below 20°C, with positive effects on immune function, blood circulation, metabolic rate, inflammation, and muscle soreness.Thu, 12 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-45/https://wzk.icu/en/blog/java-45/In-depth analysis of Nginx underlying process mechanism covering Master Worker mechanism principles and commonly used commands with practical examples.Thu, 12 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-03-programmer-salary-2/https://wzk.icu/en/blog/ai-investigation-03-programmer-salary-2/Reasons for stagnant income despite accumulated technical capabilities include position saturation, increased talent supply.Wed, 11 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-42/https://wzk.icu/en/blog/java-42/In-depth guide to Nginx configuration covering nginx.conf structure, Events block, HTTP block, reverse proxy, and load balancing with practical examples.Wed, 11 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/ai-investigation-02-programmer-salary/https://wzk.icu/en/blog/ai-investigation-02-programmer-salary/Technical investment and income are generally positively correlated, but returns do not grow infinitely.Tue, 10 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-41/https://wzk.icu/en/blog/java-41/In-depth introduction to Nginx covering its origins, development history, common scenarios, and quick configuration guide with practical examples.Tue, 10 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-44/https://wzk.icu/en/blog/java-44/In-depth guide to Spring declarative transaction support covering transaction configuration with XML mode and XML+annotation mode with practical examples.Mon, 09 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-36/https://wzk.icu/en/blog/java-36/In-depth introduction to Spring AOP aspect enhancement covering core concepts, advice types, XML+annotation approaches with code examplesThu, 05 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/java-38/https://wzk.icu/en/blog/java-38/In-depth introduction to Spring declarative transaction support covering transaction control, concepts, four characteristics, and isolation levels with practical examples.Thu, 05 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-278-spark-mllib-gbdt-case/https://wzk.icu/en/blog/bigdata-278-spark-mllib-gbdt-case/GBDT practical case study walking through the complete process from residual calculation to regression tree construction and iterative training.Wed, 04 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-276-spark-mllib-bagging-boosting-gbdt/https://wzk.icu/en/blog/bigdata-276-spark-mllib-bagging-boosting-gbdt/Introduces the differences between Bagging and Boosting in machine learning, and the GBDT (Gradient Boosting Decision Tree) algorithm principles.Tue, 03 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-277-spark-mllib-gradient-boosting/https://wzk.icu/en/blog/bigdata-277-spark-mllib-gradient-boosting/This article introduces the principles and applications of gradient boosting tree (GBDT) algorithm.Tue, 03 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-275-spark-mllib-ensemble-learning/https://wzk.icu/en/blog/bigdata-275-spark-mllib-ensemble-learning/This article systematically introduces ensemble learning methods in machine learning.Mon, 02 Jun 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-274-spark-mllib-pruning/https://wzk.icu/en/blog/bigdata-274-spark-mllib-pruning/This article systematically introduces decision tree pre-pruning and post-pruning principles, compares core differences between three mainstream algorithms.Thu, 29 May 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-273-spark-mllib-decision-tree/https://wzk.icu/en/blog/bigdata-273-spark-mllib-decision-tree/This article introduces the basic concepts, classification principles, and classification principles of decision trees.Wed, 28 May 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-272-spark-mllib-logistic-regression/https://wzk.icu/en/blog/bigdata-272-spark-mllib-logistic-regression/This article introduces the basic principles, application scenarios, and implementation in Spark MLlib of logistic regression.Tue, 27 May 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-271-spark-mllib-linear-regression/https://wzk.icu/en/blog/bigdata-271-spark-mllib-linear-regression/Linear regression uses regression equations to model relationships between independent and dependent variables.Fri, 11 Apr 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-268-ods-kafka-dim/https://wzk.icu/en/blog/bigdata-268-ods-kafka-dim/Kafka is a distributed streaming platform for high-throughput message passing. In ETL processes, Kafka serves as a data message queue or stream processing source.Fri, 03 Jan 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-269-dim-dw-ads/https://wzk.icu/en/blog/bigdata-269-dim-dw-ads/Original MySQL area table to HBase: Convert area table to region ID, region name, city ID, city name, province ID, province name, and write to HBase.Fri, 03 Jan 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-271-spark-mllib-logistic-regression/https://wzk.icu/en/blog/bigdata-271-spark-mllib-logistic-regression/Logistic Regression is a classification model in machine learning. Despite having "regression" in its name, it is a classification algorithm.Fri, 03 Jan 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-266-canal-kafka/https://wzk.icu/en/blog/bigdata-266-canal-kafka/This article introduces Alibaba's open-source Canal tool, which implements Change Data Capture (CDC) by parsing MySQL binlog.Thu, 02 Jan 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-267-lambda-kappa-architecture/https://wzk.icu/en/blog/bigdata-267-lambda-kappa-architecture/In internet companies, common ODS data includes business log data (Log) and business DB data.Thu, 02 Jan 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-270-spark-mllib-linear-regression/https://wzk.icu/en/blog/bigdata-270-spark-mllib-linear-regression/Linear Regression is an analytical method that uses regression equations to model the relationship between one or more independent variables and a dependent.Thu, 02 Jan 2025 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-265-canal-deploy/https://wzk.icu/en/blog/bigdata-265-canal-deploy/Canal is an open-source data synchronization tool from Alibaba for MySQL database incremental log parsing and synchronization.Tue, 31 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-263-canal-workflow/https://wzk.icu/en/blog/bigdata-263-canal-workflow/Canal is an open-source tool for MySQL database binlog incremental subscription and consumption.Mon, 30 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-264-mysql-binlog-research/https://wzk.icu/en/blog/bigdata-264-mysql-binlog-research/MySQL's Binary Log (binlog) is a log file type in MySQL that records all change operations performed on the database (excluding SELECT and SHOW queries).Mon, 30 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-262-canal-sync-intro/https://wzk.icu/en/blog/bigdata-262-canal-sync-intro/Alibaba B2B's cross-region business between domestic sellers and overseas buyers drove the need for data synchronization between Hangzhou and US data centers.Sun, 29 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-40/https://wzk.icu/en/blog/java-40/In-depth introduction to Spring AOP aspect enhancement covering core concepts, related terminology, and proxy configuration with practical examples.Sun, 29 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-261-realtime-warehouse-table-structure/https://wzk.icu/en/blog/bigdata-261-realtime-warehouse-table-structure/Realtime data warehouse is a data warehouse system that differs from traditional batch processing data warehouses by emphasizing low latency, high throughput.Sat, 28 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-39/https://wzk.icu/en/blog/java-39/In-depth analysis of Spring IoC container system covering circular dependency, prototype beans, prototype scope, and lazy ObjectFactory with solutions.Sat, 28 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-260-realtime-warehouse/https://wzk.icu/en/blog/bigdata-260-realtime-warehouse/Real-time data processing capability has become a key competitive factor for enterprises.Fri, 27 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-37/https://wzk.icu/en/blog/java-37/In-depth analysis of Spring IoC container system covering BeanFactory process analysis and Bean lazy loading mechanism with practical examples.Fri, 27 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-259-griffin-config/https://wzk.icu/en/blog/bigdata-259-griffin-config/Apache Griffin is an open-source data quality management framework designed to help organizations monitor and improve data quality in big data environments.Wed, 25 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-34/https://wzk.icu/en/blog/java-34/In-depth analysis of Spring IoC container initialization flow through breakpoint debugging, detailed explanation of Bean creation timing and AbstractApplicationContext.refresh core logic.Wed, 25 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-258-griffin-livy/https://wzk.icu/en/blog/bigdata-258-griffin-livy/Livy is a REST interface for Apache Spark designed to simplify Spark job submission and management, especially in big data processing scenarios.Tue, 24 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-35/https://wzk.icu/en/blog/java-35/Detailed explanation of Spring IoC container architecture, complete Bean lifecycle, and execution timing of BeanPostProcessor and BeanFactoryPostProcessor.Tue, 24 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-257-data-quality-monitor/https://wzk.icu/en/blog/bigdata-257-data-quality-monitor/Apache Griffin is an open-source big data quality solution that supports both batch and streaming data quality detection.Mon, 23 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-30/https://wzk.icu/en/blog/java-30/In-depth explanation of the differences between FactoryBean and BeanFactory in Spring, lazy loading mechanism of beans, and usage of BeanPostProcessor and BeanFactoryPostProcessor post-processors.Mon, 23 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-100-flink-cep/https://wzk.icu/en/blog/bigdata-100-flink-cep/Flink CEP detailed explanation: pattern sequence, individual patterns, combined patterns, matching skip strategies and practical cases.Sat, 21 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-256-atlas-install/https://wzk.icu/en/blog/bigdata-256-atlas-install/Metadata (MetaData) in the narrow sense refers to data that describes data.Sat, 21 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-32/https://wzk.icu/en/blog/java-32/Detailed introduction to Spring pure annotation configuration including usage of @Configuration, @ComponentScan, @Bean, @PropertySource annotations and Web environment configuration.Sat, 21 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-255-atlas-metadata/https://wzk.icu/en/blog/bigdata-255-atlas-metadata/Metadata, in its narrowest sense, refers to data that describes other data.Fri, 20 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-31/https://wzk.icu/en/blog/java-31/Explaining Spring's XML and annotation combined configuration mode, including the correspondence between annotations and XML tags, and usage of dependency injection annotations like @Autowired, @Qu.Fri, 20 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-33/https://wzk.icu/en/blog/java-33/Detailed explanation of Spring IoC configuration in pure XML mode including three ways to create Beans, lifecycle, scope, and XML configuration for constructor injection and setter injection.Thu, 19 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-98-flink-memory/https://wzk.icu/en/blog/bigdata-98-flink-memory/Flink memory model detailed explanation: Network Buffer Pool, Task Heap, State Backend memory allocation, GC tuning and backpressure handling.Wed, 18 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-99-flink-parallelism/https://wzk.icu/en/blog/bigdata-99-flink-parallelism/Flink parallelism detailed explanation: Operator Chaining, Slot allocation strategy, parallelism settings and resource scheduling principle.Wed, 18 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-29/https://wzk.icu/en/blog/java-29/"BeanFactory is the top-level interface of the Spring framework's IoC container.Wed, 18 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-254-airflow-trade-task/https://wzk.icu/en/blog/bigdata-254-airflow-trade-task/Apache Airflow is an open-source task scheduling and workflow management tool for orchestrating complex data processing tasks.Tue, 17 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-27/https://wzk.icu/en/blog/java-27/Manually implement AOP aspect-oriented programming on top of business logic, using JDK dynamic proxies and CGLIB to implement transaction manager for automatic commit and rollback.Tue, 17 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-253-airflow-core-concept/https://wzk.icu/en/blog/bigdata-253-airflow-core-concept/Apache Airflow is an open-source task scheduling and workflow management tool for orchestrating complex data processing tasks.Mon, 16 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-24/https://wzk.icu/en/blog/java-24/Step-by-step guide to manually implement an IoC container using BeanFactory and XML configuration to manage Bean dependencies and complete core dependency injection implementation.Mon, 16 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-252-airflow-crontab/https://wzk.icu/en/blog/bigdata-252-airflow-crontab/Linux systems use the cron (crond) service for scheduled tasks, which is enabled by default.Sun, 15 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-178/https://wzk.icu/en/blog/java-178/FastDFS architecture design: lightweight Tracker, peer storage structure, file upload and sync principles. This article analyzes principles and practices to help readers master key technical points.Sun, 15 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-250-ads-airflow/https://wzk.icu/en/blog/bigdata-250-ads-airflow/Apache Airflow is an open-source task scheduling and workflow management platform, primarily used for developing, debugging, and monitoring data pipelines.Sat, 14 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-251-airflow-install/https://wzk.icu/en/blog/bigdata-251-airflow-install/Apache Airflow is an open-source task scheduling and workflow management tool for orchestrating complex data processing tasks.Sat, 14 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-96-flink-broadcast-state/https://wzk.icu/en/blog/bigdata-96-flink-broadcast-state/Flink Broadcast State explanation: BroadcastState principle, dynamic rule updates, state partitioning and memory management, demonstrating broadcast stream and non-broadcast stream join through cases.Sat, 14 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-97-flink-state-backend/https://wzk.icu/en/blog/bigdata-97-flink-state-backend/Flink State Backend detailed explanation: HashMapStateBackend, EmbeddedRocksDBStateBackend selection, memory configuration and performance tuning.Sat, 14 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-28/https://wzk.icu/en/blog/java-28/Manually implement business logic layering without using Spring, including DAO layer, Service layer, and Controller layer, demonstrating dependency management issues in traditional development.Sat, 14 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-26/https://wzk.icu/en/blog/java-26/Step-by-step guide to implementing a simple Spring IoC container, Part 1 covering Servlet basics, characteristics, working principles, and writing your first Servlet program.Fri, 13 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-247-hive-order-zipper-table/https://wzk.icu/en/blog/bigdata-247-hive-order-zipper-table/This article continues the zipper table practice, focusing on order history state incremental refresh.Thu, 12 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-248-dimension-tables/https://wzk.icu/en/blog/bigdata-248-dimension-tables/First determines fact tables vs dimension tables: green indicates fact tables, gray indicates dimension tables.Thu, 12 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-249-dwd-dws-layer/https://wzk.icu/en/blog/bigdata-249-dwd-dws-layer/The order table is a periodic fact table; zipper tables can be used to retain order status history. Order detail tables are regular fact tables.Thu, 12 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-174/https://wzk.icu/en/blog/java-174/FastDFS core architecture deep dive: Tracker Server, Storage Server, Group mechanism. This article analyzes principles and practices to help readers master key technical points.Thu, 12 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-25/https://wzk.icu/en/blog/java-25/Comprehensive introduction to Spring framework basics, evolution from EJB to Spring, detailed explanation of Spring's core concepts including IoC and AOP with practical implementation analysis.Thu, 12 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-242-datax-mysql-hdfs-hive/https://wzk.icu/en/blog/bigdata-242-datax-mysql-hdfs-hive/This article provides a practical guide to Hive zipper tables for preserving order history states at low cost in offline data warehouses, supporting daily backtracking and change analysis.Wed, 11 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-23/https://wzk.icu/en/blog/java-23/Optimize handwritten ORM framework, focusing on getMapper method implementation in DefaultSqlSession, using JDK dynamic proxy to create Mapper interface proxy objects.Wed, 11 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-246-hive-zipper-table-advanced/https://wzk.icu/en/blog/bigdata-246-hive-zipper-table-advanced/userinfo (partitioned table) => userid, mobile, regdate => Daily changed data (modified + new) / Historical data (first day) userhis (zipper table) => Two additional fields: startdate / enddateTue, 10 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-177/https://wzk.icu/en/blog/java-177/Complete guide for FastDFS 6.15.0 source compilation and Nginx fastdfs-nginx-module configuration. This article analyzes principles and practices to help readers master key technical points.Tue, 10 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-22/https://wzk.icu/en/blog/java-22/In-depth analysis of SqlSession and Executor working principles in handwritten ORM framework, detailing implementation mechanisms of SqlSessionFactoryBuilder, DefaultSqlSession, SimpleExecutor.Tue, 10 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-245-hive-scd-zipper-table/https://wzk.icu/en/blog/bigdata-245-hive-scd-zipper-table/Slowly Changing Dimensions (SCD) refer to dimension attributes that change slowly over time in the real world (slow is relative to fact tables, where data changes faster than dimension tables).Mon, 09 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-21/https://wzk.icu/en/blog/java-21/In the current project, create under resources: sqlMapConfig.xml mapper.xmlMon, 09 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-176/https://wzk.icu/en/blog/java-176/FastDFS distributed file system architecture, evolution from standalone to distributed. This article analyzes principles and practices to help readers master key technical points and practical skills.Sun, 08 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-20/https://wzk.icu/en/blog/java-20/First article in the handwritten ORM framework series, introducing problems and disadvantages of traditional JDBC development and ORM framework design ideas and solutions.Sun, 08 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-244-hive-ods-layer/https://wzk.icu/en/blog/bigdata-244-hive-ods-layer/Sync MySQL data to specified HDFS directory via DataX, then create ODS external tables in Hive with unified dt string partitioning.Sat, 07 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-243-datax-incremental-import/https://wzk.icu/en/blog/bigdata-243-datax-incremental-import/Scenario: Three core e-commerce transaction tables do daily incremental to offline data warehouse ODS, partitioned by dt Conclusion: DataX uses MySQLReader + HDFSWriter, extracts by time field, HDFS directory partitioned...Fri, 06 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-16/https://wzk.icu/en/blog/java-16/Proxy Pattern: Provides a proxy for an object and controls access to the original object through the proxy. The English name of proxy pattern is Proxy.Fri, 06 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-170/https://wzk.icu/en/blog/java-170/Complete guide to integrating Spring Boot with Neo4j: Java Driver, Neo4jTemplate, Repository pattern with practical examples for graph database CRUD operations and relationship queries.Thu, 05 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-18/https://wzk.icu/en/blog/java-18/Detailed introduction to design patterns used in MyBatis source code including builder pattern, factory method pattern, singleton pattern, proxy pattern, composite pattern, and their manifestation.Thu, 05 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-241-mysql-source-table-design/https://wzk.icu/en/blog/bigdata-241-mysql-source-table-design/Focusing on three main metrics: order count, product count, payment amount, breakdown analysis dimensions by sales region and product type (3-level category).Wed, 04 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-95-flink-state/https://wzk.icu/en/blog/bigdata-95-flink-state/Flink stateful computation explanation: Keyed State, Operator State, Checkpoint configuration, Savepoint backup and recovery, production environment practices.Wed, 04 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-240-hive-ads-datax-export-mysql-advertising/https://wzk.icu/en/blog/bigdata-240-hive-ads-datax-export-mysql-advertising/Complete solution for exporting Hive ADS layer data to MySQL using DataX.Tue, 03 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-171/https://wzk.icu/en/blog/java-171/Neo4j embedded database vs server mode comparison, Java API access examples.Tue, 03 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-239-flume-hdfs-hive-ods-dwd-advertising/https://wzk.icu/en/blog/bigdata-239-flume-hdfs-hive-ods-dwd-advertising/Using Flume Agent to collect event logs and write to HDFS, then use Hive scripts to complete ODS and DWD layer data loading by date.Mon, 02 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-172/https://wzk.icu/en/blog/java-172/Neo4j database backup and recovery, data warm-up and execution plan analysis.Sun, 01 Dec 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-238-hive-advertising-ads-ctr-cvr-top100/https://wzk.icu/en/blog/bigdata-238-hive-advertising-ads-ctr-cvr-top100/action: User behavior; 0 impression; 1 click after impression; 2 purchase duration: Stay duration shopid: Merchant id eventtype: "ad" adtype: Format type; 1 JPG; 2 PNGSat, 30 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-93-flink-streaming-intro/https://wzk.icu/en/blog/bigdata-93-flink-streaming-intro/This is article 93 in the Big Data series, introducing Flink DataStream API core concepts and program structure.Sat, 30 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-94-flink-window/https://wzk.icu/en/blog/bigdata-94-flink-window/Comprehensive analysis of Flink Window mechanism: tumbling windows, sliding windows, session windows, Watermark principle and generation strategies, late data processing mechanism.Sat, 30 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-237-hive-advertising-ods-dwd-event-parsing/https://wzk.icu/en/blog/bigdata-237-hive-advertising-ods-dwd-event-parsing/This article introduces completing parsing, cleaning, and detail modeling from ODS to DWD for offline data warehouse based on advertising events in tracking logs.Fri, 29 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-236-member-metrics-datax-ads-advertising/https://wzk.icu/en/blog/bigdata-236-member-metrics-datax-ads-advertising/This is a practical verification article for member theme and advertising business pipeline based on Hadoop + Hive + HDFS + DataX + MySQL.Thu, 28 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-173/https://wzk.icu/en/blog/java-173/Neo4j transaction handling, index creation, constraint settings and concurrency issue troubleshooting.Thu, 28 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-235-flume-hdfs-hive-ods-dwd-dws-ads/https://wzk.icu/en/blog/bigdata-235-flume-hdfs-hive-ods-dwd-dws-ads/This article demonstrates a complete offline data warehouse pipeline from log collection to member metric analysis.Wed, 27 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-91-flink-install/https://wzk.icu/en/blog/bigdata-91-flink-install/Apache Flink is a distributed stream processing framework widely used for real-time data computing scenarios.Wed, 27 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-92-flink-yarn/https://wzk.icu/en/blog/bigdata-92-flink-yarn/Detailed explanation of three Flink deployment modes on YARN cluster: Session, Application, Per-Job modes, Hadoop dependency configuration, YARN resource application and job submission process.Wed, 27 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-234-hive-ads-datax-export-mysql/https://wzk.icu/en/blog/bigdata-234-hive-ads-datax-export-mysql/The landing path for exporting Hive ADS layer tables to MySQL in offline data warehouse. Gives typical DataX solution: hdfsreader -> mysqlwriter.Tue, 26 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-11/https://wzk.icu/en/blog/java-11/The relationship between user table and order table: one user has multiple orders, one order belongs to one user.Tue, 26 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-233-hive-retention-rate-dws-ads/https://wzk.icu/en/blog/bigdata-233-hive-retention-rate-dws-ads/Implementation of 'Member Retention' in offline data warehouse: DWS layer uses dwsmemberretention_day table to join new member and startup detail tables to.Mon, 25 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-10/https://wzk.icu/en/blog/java-10/Detailed introduction to MyBatis annotation development approach, including usage of common annotations (@Select, @Insert, @Update, @Delete, @Results, etc.Mon, 25 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-169/https://wzk.icu/en/blog/java-169/Neo4j CQL functions quick reference: string functions, aggregate functions, relationship functions and shortest path queries.Mon, 25 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-232-hive-new-member-retention/https://wzk.icu/en/blog/bigdata-232-hive-new-member-retention/Offline data warehouse calculates 'new members' daily, and provides consistent definition data foundation for subsequent 'member retention'.Sat, 23 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-90-flink-intro/https://wzk.icu/en/blog/bigdata-90-flink-intro/Systematic introduction to Apache Flink's origin, core features, and architecture components: JobManager, TaskManager, Dispatcher responsibilities, unified stream-batch processing model.Sat, 23 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-08/https://wzk.icu/en/blog/java-08/In database design, a "many-to-many" relationship means records in two tables can be associated with each other, with multiple records associated with multiple records.Sat, 23 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-231-hive-dwd-dws-ads-active-member/https://wzk.icu/en/blog/bigdata-231-hive-dwd-dws-ads-active-member/This article introduces using Hive to build an offline data warehouse for calculating active members (daily/weekly/monthly).Fri, 22 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-07/https://wzk.icu/en/blog/java-07/In database design, one-to-many (One-to-Many) relationship is one of the most common relationship models.Fri, 22 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-166/https://wzk.icu/en/blog/java-166/Neo4j CQL advanced queries: WHERE clause filtering, DELETE removal, SET updates, ORDER BY sorting and pagination.Fri, 22 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-230-ods-layer-hive-json/https://wzk.icu/en/blog/bigdata-230-ods-layer-hive-json/JSON data processing in Hive offline data warehouse, covering three most common needs: 1) Extract array fields from JSON strings and explode-expand in SQLThu, 21 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-05/https://wzk.icu/en/blog/java-05/Detailed explanation of one-to-one model mapping in MyBatis, including database table design, Java entity class definitions, Mapper interface and XML configuration.Thu, 21 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-229-ods-layer/https://wzk.icu/en/blog/bigdata-229-ods-layer/Engineering implementation of the ODS (Operational Data Store) layer in an offline data warehouse: the minimal closed loop of Hive external table + daily.Wed, 20 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-89-spark-streaming-kafka/https://wzk.icu/en/blog/bigdata-89-spark-streaming-kafka/This is article 89 in the Big Data series, deeply comparing two core modes of Spark Streaming integration with Kafka, focusing on Direct mode production practices.Wed, 20 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-04/https://wzk.icu/en/blog/java-04/In-depth explanation of MyBatis dynamic SQL configuration including if conditions, where tags, foreach loops, and sql fragment extraction.Wed, 20 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-167/https://wzk.icu/en/blog/java-167/Neo4j CQL query language guide: CREATE creates nodes and relationships, MATCH queries graph data, with complete code examples for graph database operations and traversal patterns.Wed, 20 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-17/https://wzk.icu/en/blog/java-17/Detailed introduction to MyBatis Mapper Proxy dynamic proxy implementation principles, including getMapper source code analysis and MapperMethod execution process.Wed, 20 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-228-flume-taildir-interceptor/https://wzk.icu/en/blog/bigdata-228-flume-taildir-interceptor/Apache Flume offline log collection implementation using Taildir Source and a custom Interceptor to extract JSON timestamps, mark headers, and route HDFS partitions by event time, with development and deployment notes.Tue, 19 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-06/https://wzk.icu/en/blog/java-06/In-depth explanation of MyBatis core components: SqlSessionFactory and SqlSession working principles, traditional DAO layer implementation, and MyBatis proxy development mode.Tue, 19 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-227-flume-custom-interceptor/https://wzk.icu/en/blog/bigdata-227-flume-custom-interceptor/Using TAILDIR Source to monitor multiple directories (start/event), using filegroups headers to mark different sources with logtypeMon, 18 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-01/https://wzk.icu/en/blog/java-01/MyBatis quick start continuation covering CRUD operations, XML mapping configuration, annotation-based development, and datasource/transaction management.Mon, 18 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-168/https://wzk.icu/en/blog/java-168/Neo4j graph database Docker installation and remote access configuration, 10-minute quick start guide.Mon, 18 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-19/https://wzk.icu/en/blog/java-19/2k, likes: 27, favorites: 13. Configuration comes from two sources: configuration files (main config.xml, mapper.xml) and Java code annotations.Mon, 18 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-14/https://wzk.icu/en/blog/java-14/Detailed introduction to MyBatis pagination plugin PageHelper and Generic Mapper usage, including configuration, code examples, PageInfo class usage, and Generic Mapper CRUD operations.Sun, 17 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-226-flume-optimization/https://wzk.icu/en/blog/bigdata-226-flume-optimization/Flume 1.9.0 tuning guide for offline data warehouse log collection to HDFS, covering batch parameters, channel capacity and transaction sizing, JVM heap tuning, and interceptor-based event-time partition correction.Sat, 16 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-87-spark-dstream-transform/https://wzk.icu/en/blog/bigdata-87-spark-dstream-transform/Systematically review Spark Streaming DStream stateless transformation operators and transform advanced operations, demonstrating three implementation approaches for blacklist filtering: leftOuterJ.Sat, 16 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-88-spark-streaming-window/https://wzk.icu/en/blog/bigdata-88-spark-streaming-window/In-depth explanation of Spark Streaming stateful computing: window operation parameter configuration, reduceByKeyAndWindow hot word statistics, updateStateByKey full-state maintenance and mapWithSt.Sat, 16 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-03/https://wzk.icu/en/blog/java-03/MyBatis quick start tutorial covering MyBatis development steps including POM dependency configuration, Mapper mapping files, SqlMapConfig core configuration file, and CRUD operations.Sat, 16 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-225-offline-dw-member-metrics/https://wzk.icu/en/blog/bigdata-225-offline-dw-member-metrics/Scenario: Use startup logs/event logs in offline data warehouse to count new, active (DAU/WAU/MAU), retention.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-02/https://wzk.icu/en/blog/java-02/ORM stands for Object/Relation Mapping: an abbreviation for object-relation mapping.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-101/https://wzk.icu/en/blog/java-101/This article provides an in-depth explanation of MySQL master-slave replication: master-slave mode, read-write separation, semi-sync replication principles and configuration.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-102/https://wzk.icu/en/blog/java-102/This article provides an in-depth explanation of MySQL database design: availability (redundancy, failover), scalability (sharding, read-write separation), consistency.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-103/https://wzk.icu/en/blog/java-103/This article provides an in-depth explanation of MySQL deadlocks: deadlock definition, necessary conditions, table lock deadlocks, row-level deadlocks, shared to exclusive lock conversion.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-104/https://wzk.icu/en/blog/java-104/Pessimistic locking is a concurrency control mechanism based on a conservative "lock first, then modify" strategy.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-105/https://wzk.icu/en/blog/java-105/This article provides an in-depth explanation of MySQL parallel replication technology: from 5.6 database-based parallel to 5.7 group commit-based to 8.0 writeset-based evolution.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-107/https://wzk.icu/en/blog/java-107/MySQL dual-master mode is a common high-availability database architecture solution, particularly suitable for the following business scenarios:Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-109/https://wzk.icu/en/blog/java-109/Deep dive into MySQL MHA high availability solution: MHA architecture, fault handling, primary-standby switch, lag issues and solutions. This article analyzes principles and practical applications.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-111/https://wzk.icu/en/blog/java-111/Sharding is the core technology for horizontal scaling in distributed databases.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-112/https://wzk.icu/en/blog/java-112/Explore MySQL distributed primary key strategies including UUID, SnowFlake, COMB, Redis, and database ID table approaches with their principles, pros and cons, and applicable scenarios comparison.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-113/https://wzk.icu/en/blog/java-113/Explore MySQL database horizontal scaling solutions for capacity, performance, and concurrency bottlenecks.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-114/https://wzk.icu/en/blog/java-114/A comprehensive guide to Sharding-JDBC, a lightweight Java database middleware solving database sharding, read-write separation and distributed transactions.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-115/https://wzk.icu/en/blog/java-115/In distributed databases, sharding is implemented through physical tables, logical tables, data nodes and binding tables.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-116/https://wzk.icu/en/blog/java-116/Deep dive into ShardingSphere's sharding flow including SQL parsing, query optimization, SQL routing, SQL rewriting, SQL execution and result merging six major stages with detailed technical analysis.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-117/https://wzk.icu/en/blog/java-117/Read-write separation is a common database architecture optimization technique that separates write operations to master and read operations to slaves through master-slave replication.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-118/https://wzk.icu/en/blog/java-118/Deep dive into ShardingSphere's SQL support scope, limitations, and optimization practices in sharding scenarios. Learn about supported SQL types, pagination subquery restrictions, and best practices.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-119/https://wzk.icu/en/blog/java-119/In distributed databases, sharding uses physical tables, logical tables, data nodes and binding tables for efficient data storage and queries.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-120/https://wzk.icu/en/blog/java-120/In distributed database scenarios, ShardingSphere provides the Hint mechanism to force routing by explicitly specifying sharding values outside SQL, bypassing the default sharding strategy and directing statements to tar...Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-121/https://wzk.icu/en/blog/java-121/Distributed transaction patterns mainly include TCC, message queue, and Saga.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-122/https://wzk.icu/en/blog/java-122/Seata is Alibaba's open-source distributed transaction solution providing one-stop transaction management with high performance, low intrusion and multi-language support.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-123/https://wzk.icu/en/blog/java-123/Traditional approaches rely on database vendor XA drivers and application server transaction managers. Modern systems use embedded transaction managers.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-124/https://wzk.icu/en/blog/java-124/CAP theory reveals that distributed systems cannot simultaneously satisfy consistency, availability and partition tolerance.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-125/https://wzk.icu/en/blog/java-125/Data masking is a key data security technology that balances data utility and privacy protection by replacing, encrypting or perturbing sensitive information.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-126/https://wzk.icu/en/blog/java-126/ShardingSphere's flexible transactions are implemented through the third-party servicecomb-saga component, using SPI mechanism for dynamic injection.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-127/https://wzk.icu/en/blog/java-127/MyCat is an open-source distributed database middleware fully compatible with MySQL protocol.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-128/https://wzk.icu/en/blog/java-128/Sharding-Proxy is one of the core products in the Apache ShardingSphere ecosystem, using a transparent database proxy architecture.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-129/https://wzk.icu/en/blog/java-129/MyCat distributed database sharding strategies include E-R table sharding, global table mechanism, shard nodes, node hosts and sharding rules.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-130/https://wzk.icu/en/blog/java-130/server.xml is MyCat's core configuration file, storing almost all system configuration information needed by MyCat.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-131/https://wzk.icu/en/blog/java-131/MyCAT is an open-source database middleware supporting MySQL protocol, commonly used for database sharding and data sharding management.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-132/https://wzk.icu/en/blog/java-132/MySQL transaction isolation levels include READ UNCOMMITTED, READ COMMITTED, REPEATABLE READ and SERIALIZABLE. Each level solves different concurrency problems.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-133/https://wzk.icu/en/blog/java-133/MySQL database transaction is a set of SQL operations executed as a single logical unit of work, following ACID principles: Atomicity, Consistency, Isolation and Durability.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-134/https://wzk.icu/en/blog/java-134/schema.xml is MyCat's core configuration file managing logical databases, logical tables, data nodes and data hosts.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-135/https://wzk.icu/en/blog/java-135/Common issues in database transactions include dirty read, non-repeatable read, and phantom read. These issues are closely related to database isolation levels.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-136/https://wzk.icu/en/blog/java-136/Transaction propagation is an important mechanism in Spring for handling transaction boundaries, defining how methods interact with transaction context in the call chain.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-137/https://wzk.icu/en/blog/java-137/Introduces declarative transaction and programmatic transaction concepts, usage methods and comparative analysis.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-138/https://wzk.icu/en/blog/java-138/Spring transactions are implemented through AOP dynamic proxies. JDK dynamic proxy requires the target method to be public.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-139/https://wzk.icu/en/blog/java-139/Detailed introduction to @Transactional annotation parameters in Spring Boot, including transaction propagation, isolation levels, and exception rollback strategies.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-13/https://wzk.icu/en/blog/java-13/Generally, open-source frameworks provide plugins or extension points for developers to extend on their own.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-140/https://wzk.icu/en/blog/java-140/Spring transaction pitfalls commonly occur in scenarios like classes not managed by Spring, transaction manager not started, improper method modifiers (final, static, private).Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-141/https://wzk.icu/en/blog/java-141/Spring transaction pitfalls include rollbackFor configuration errors, transaction annotation overrides, nested transaction issues and more.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-142/https://wzk.icu/en/blog/java-142/MongoDB common commands include show dbs to list databases, use to switch databases, db.createCollection() to create collections, show collections to list collections, db.collection.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-143/https://wzk.icu/en/blog/java-143/NoSQL is a general term for non-relational databases, including document-based MongoDB, key-value Redis, column-store HBase, and graph database Neo4j, each with its own characteristics and use cases.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-144/https://wzk.icu/en/blog/java-144/BSON is MongoDB's core storage format, a binary-encoded JSON-like format. Compared to JSON, BSON supports more data types like Date, Binary, ObjectId, with better performance.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-145/https://wzk.icu/en/blog/java-145/Spring transaction pitfalls include multi-threaded calls, exception catching without rollback, manual exception throwing errors.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-146/https://wzk.icu/en/blog/java-146/MongoDB indexes are core mechanisms for improving query performance, including single field indexes, compound indexes, multi-key indexes, geospatial indexes, text indexes and hashed indexes.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-147/https://wzk.icu/en/blog/java-147/MongoDB index management includes createIndex, getIndexes, dropIndex operations.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-148/https://wzk.icu/en/blog/java-148/MongoDB aggregation framework processes documents through multi-stage pipelines, implementing complex operations similar to SQL GROUP BY, JOIN, SUM, AVG.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-149/https://wzk.icu/en/blog/java-149/MongoDB implements pagination through combination of skip(), limit() and sort() methods.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-150/https://wzk.icu/en/blog/java-150/MongoDB data operations include single insertOne() and batch insertMany(). Query operators include $eq, $ne, $gt, $lt, etc. Logical operators include $and, $or, $not.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-151/https://wzk.icu/en/blog/java-151/MongoDB uses B-tree indexes where nodes store both data and keys. MySQL uses B+tree indexes with all data concentrated in leaf nodes.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-152/https://wzk.icu/en/blog/java-152/MongoDB is suitable for website data storage, caching systems, big data analysis, IoT, social media, live streaming platforms and more.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-153/https://wzk.icu/en/blog/java-153/Two ways to connect Spring Boot to MongoDB: MongoTemplate and MongoRepository, with complete Java code examples and common problem troubleshooting.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-154/https://wzk.icu/en/blog/java-154/Complete guide to accessing MongoDB with Java, including environment setup, Maven dependencies, connection, CRUD operations, POJO mapping, index management.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-155/https://wzk.icu/en/blog/java-155/MongoDB slow query analysis through built-in Profiler, set db.setProfilingLevel(1, m) to record slow queries. EXPLAIN analysis helps identify performance bottlenecks and optimize queries.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-156/https://wzk.icu/en/blog/java-156/Comparison of MongoDB's two main storage engines WiredTiger and InMemory, including configuration methods and verification steps. WiredTiger recommended for production environments.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-157/https://wzk.icu/en/blog/java-157/MongoDB 7 replica set containerization best practices: generate keyfile, explicitly enable --auth, use one-time container for rs.initiate and admin user creation.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-158/https://wzk.icu/en/blog/java-158/MongoDB replica set single machine three instance deployment guide including configuration, initialization, adding/removing nodes, elections, read-write separation and rollback operations.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-159/https://wzk.icu/en/blog/java-159/MongoDB data modeling choice between embedded and reference, following three principles: embed for 1:few, use reference for unbounded growth, use $lookup+index for cross-collection association.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-160/https://wzk.icu/en/blog/java-160/MongoDB replica set concepts: oplog location and idempotency, heartbeat frequency and election timeout, member/voting limits (50/7). Note that master-slave replication was removed since 4.0.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-161/https://wzk.icu/en/blog/java-161/MongoDB production environment security hardening guide including authentication, RBAC minimum privileges, internal network bindIp/firewall ACL, TLS/SSL encryption.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-162/https://wzk.icu/en/blog/java-162/MongoDB sharded cluster detailed explanation: sharding consists of Shard, Config Server, Mongos. Shard key selection includes range sharding, hashed sharding, zone sharding.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-163/https://wzk.icu/en/blog/java-163/MongoDB authentication and permission configuration from single instance to sharded cluster. Implementation order: create admin/root first → enable authorization → assign permissions by database.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-164/https://wzk.icu/en/blog/java-164/Step-by-step MongoDB sharded cluster setup (configsvr×3 + two shard replica sets×3 nodes + multiple mongos), including configuration, initialization, adding shards, balancing and rollback.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-165/https://wzk.icu/en/blog/java-165/Euler path/circuit determination: undirected graphs check connectivity and odd-degree vertices, directed graphs check weak connectivity and in/out-degree balance.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-96/https://wzk.icu/en/blog/java-96/This article provides an in-depth explanation of MySQL transaction isolation levels: Read Uncommitted, Read Committed, Repeatable Read, and Serializable, analyzing dirty read.Fri, 15 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-223-offline-dw-building/https://wzk.icu/en/blog/bigdata-223-offline-dw-building/Complete guide to building offline data warehouses: data tracking, metric system design, theme analysis, and standardization practices for e-commerce teams.Thu, 14 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-224-offline-dw-architecture/https://wzk.icu/en/blog/bigdata-224-offline-dw-architecture/Offline Data Warehouse (Offline DW) overall architecture design and implementation method: Framework selection comparison between Apache community version and.Thu, 14 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-12/https://wzk.icu/en/blog/java-12/Currently, the level 2 cache we implement is per service. At this time, the cache is on a single server.Thu, 14 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-221-offline-dw-layers/https://wzk.icu/en/blog/bigdata-221-offline-dw-layers/Scenario: The more department-built data marts, the more inconsistent definitions, disconnected interfaces, forming data silos, and exploding data query costs.Wed, 13 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-222-offline-dw-modeling/https://wzk.icu/en/blog/bigdata-222-offline-dw-modeling/In data warehouse architecture, Fact Table is the core table structure that stores business process metric values or facts.Wed, 13 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-85-spark-streaming-intro/https://wzk.icu/en/blog/bigdata-85-spark-streaming-intro/This is article 85 in the Big Data series, introducing the architecture and evolution background of Spark's two generations of streaming frameworks.Wed, 13 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-86-spark-streaming-sources/https://wzk.icu/en/blog/bigdata-86-spark-streaming-sources/Comprehensive explanation of three Spark Streaming basic data sources: file stream directory monitoring, Socket TCP ingestion, RDD queue stream for testing simulation.Wed, 13 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-09/https://wzk.icu/en/blog/java-09/Detailed introduction to MyBatis level 1 cache working principles, code testing, invalidation scenarios, and source code analysis.Wed, 13 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/java-15/https://wzk.icu/en/blog/java-15/Detailed introduction to MyBatis level 2 cache working principles, enable configuration, code testing, and source code analysis.Wed, 13 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-219-grafana-install/https://wzk.icu/en/blog/bigdata-219-grafana-install/For OPs/devs still using CentOS/RHEL (including compatible distributions) in 2026, provides Grafana 11.3.0 (grafana-enterprise-11.3.0-1.x86_64.Tue, 12 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-220-data-warehouse-intro/https://wzk.icu/en/blog/bigdata-220-data-warehouse-intro/In 1988, IBM first introduced the concept of "Information Warehouse" when facing increasingly scattered enterprise information systems and growing data silo problems.Tue, 12 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-217-prometheus-install/https://wzk.icu/en/blog/bigdata-217-prometheus-install/Scenario: Single-machine deployment of Prometheus 2.53.2, pull node_exporter metrics from multiple hosts and verify Targets status.Mon, 11 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-218-prometheus-exporter/https://wzk.icu/en/blog/bigdata-218-prometheus-exporter/Common Prometheus monitoring deployment: Install node_exporter-1.8.2 on Rocky Linux to expose host metrics, integrate with Prometheus scrape config, and visualize in Grafana dashboards.Mon, 11 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-215-kmeans-evaluation/https://wzk.icu/en/blog/bigdata-215-kmeans-evaluation/Scenario: Using sklearn for KMeans clustering, want to explain centroids/loss and use metrics for K selection.Sat, 09 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-216-kmeans-nclusters-silhouette/https://wzk.icu/en/blog/bigdata-216-kmeans-nclusters-silhouette/KMeans nclusters selection method: calculate silhouettescore and silhouette_samples on candidate cluster numbers (e.g.Sat, 09 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-83-sparksql-hive/https://wzk.icu/en/blog/bigdata-83-sparksql-hive/Comprehensive guide to SparkSQL core usage including DataFrame API operations, SQL query syntax, lateral view explode, and Hive integration via enableHiveSupport for metadata and table operations.Sat, 09 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-84-sparksql-optimizer/https://wzk.icu/en/blog/bigdata-84-sparksql-optimizer/This is article 84 in the Big Data series, deeply analyzing SparkSQL kernel's Join strategy auto-selection logic and SQL parsing optimization flow.Sat, 09 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-213-kmeans-iris-python/https://wzk.icu/en/blog/bigdata-213-kmeans-iris-python/Scenario: Hand-write K-Means using NumPy/Pandas, perform 3-class clustering on Iris.txt and output centroids with clustering results.Fri, 08 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-214-kmeans-sklearn/https://wzk.icu/en/blog/bigdata-214-kmeans-sklearn/K-Means clustering provides an engineering workflow that is 'verifiable, reproducible, and debuggable': first use 2D testSet dataset for algorithm verification.Fri, 08 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-211-logistic-regression-sklearn/https://wzk.icu/en/blog/bigdata-211-logistic-regression-sklearn/When using Logistic Regression in Scikit-Learn, max_iter controls maximum iterations affecting model convergence speed and accuracy.Thu, 07 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-212-kmeans-intro/https://wzk.icu/en/blog/bigdata-212-kmeans-intro/K-Means clustering algorithm, comparing supervised vs unsupervised learning (whether labels Y are needed).Thu, 07 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-209-logistic-regression-gradient-descent/https://wzk.icu/en/blog/bigdata-209-logistic-regression-gradient-descent/Logistic Regression (LR) is an important classification algorithm in machine learning.Wed, 06 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-210-logistic-regression-sklearn-l1-l2/https://wzk.icu/en/blog/bigdata-210-logistic-regression-sklearn-l1-l2/As C gradually increases, regularization strength gets smaller, model performance on training and test shows upward trend, until around C=0.8.Wed, 06 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-81-sparksql-rdd-df-ds/https://wzk.icu/en/blog/bigdata-81-sparksql-rdd-df-ds/This is article 81 in the Big Data series, comprehensively introducing Spark's three core data abstractions' features, use cases and mutual conversions.Wed, 06 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-82-sparksql-operations/https://wzk.icu/en/blog/bigdata-82-sparksql-operations/This is article 82 in the Big Data series, systematically introducing SparkSQL Transformation and Action operators with complete test cases.Wed, 06 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-207-multicollinearity/https://wzk.icu/en/blog/bigdata-207-multicollinearity/When using scikit-learn for linear regression, how to handle multicollinearity in least squares method.Tue, 05 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-208-ridge-lasso-regression/https://wzk.icu/en/blog/bigdata-208-ridge-lasso-regression/Ridge Regression and Lasso Regression are two commonly used linear regression regularization methods for solving overfitting and multicollinearity in machine.Tue, 05 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-205-linear-regression-matrix-sse/https://wzk.icu/en/blog/bigdata-205-linear-regression-matrix-sse/Linear Regression core chain: unify prediction function y=Xw in matrix form, treat parameter vector w as only unknownMon, 04 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-206-numpy-linear-regression/https://wzk.icu/en/blog/bigdata-206-numpy-linear-regression/pandas DataFrame and NumPy matrix multiplication hand-written multivariate linear regression (linear regression implementation).Mon, 04 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-203-sklearn-decision-tree-pruning-params/https://wzk.icu/en/blog/bigdata-203-sklearn-decision-tree-pruning-params/Common parameters for decision tree pruning (pre-pruning) in engineering: maxdepth, minsamplesleaf, minsamplessplit, maxfeatures, minimpuritydecrease.Sat, 02 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-204-confusion-matrix-roc/https://wzk.icu/en/blog/bigdata-204-confusion-matrix-roc/Confusion matrix (TP, FP, FN, TN) with unified metrics: Accuracy, Precision, Recall (Sensitivity), F1 Measure, ROC curve, AUC value, and practical business interpretation for classification models.Sat, 02 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-79-spark-standalone/https://wzk.icu/en/blog/bigdata-79-spark-standalone/Comprehensive explanation of Spark Standalone cluster four core components, application submission flow, SparkContext internal architecture, Shuffle evolution history and RDD optimization strategies.Sat, 02 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-80-sparksql-intro/https://wzk.icu/en/blog/bigdata-80-sparksql-intro/Systematic introduction to SparkSQL evolution history, core abstractions DataFrame/Dataset, Catalyst optimizer principle, and practical usage of multi-data source integration with Hive/HDFS.Sat, 02 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-201-decision-tree-split-pruning/https://wzk.icu/en/blog/bigdata-201-decision-tree-split-pruning/Decision tree is a tree-structured supervised learning model, commonly used for classification and regression tasks.Fri, 01 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-202-sklearn-decision-tree-practice/https://wzk.icu/en/blog/bigdata-202-sklearn-decision-tree-practice/Complete flow of DecisionTreeClassifier on load_wine dataset from data splitting, model evaluation to decision tree visualization (2026 version).Fri, 01 Nov 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-199-decision-tree/https://wzk.icu/en/blog/bigdata-199-decision-tree/Tree model is a widely used algorithm type in supervised learning, can be applied to both classification and regression problems.Thu, 31 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-200-decision-tree-info-gain/https://wzk.icu/en/blog/bigdata-200-decision-tree-info-gain/Scenario: Use information entropy/information gain to explain why decision tree selects certain column for splitting, and use Python to reproduce "best split column".Thu, 31 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-197-cross-validation/https://wzk.icu/en/blog/bigdata-197-cross-validation/Random train/test split causes evaluation metrics to be unstable, and gives engineering solution: K-Fold Cross Validation.Wed, 30 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-198-knn-normalization/https://wzk.icu/en/blog/bigdata-198-knn-normalization/In scikit-learn pipelines, distance-based models like KNN are highly sensitive to inconsistent feature scales. Split first, fit MinMaxScaler only on the training set, transform the test set with the same parameters...Wed, 30 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-77-spark-checkpoint/https://wzk.icu/en/blog/bigdata-77-spark-checkpoint/Detailed explanation of Spark Checkpoint execution flow, core differences with persist/cache, partitioner strategies, and best practices for iterative algorithms and long dependency chain scenarios.Wed, 30 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-78-spark-broadcast/https://wzk.icu/en/blog/bigdata-78-spark-broadcast/Detailed explanation of Spark broadcast variable working principle, configuration parameters and best practices.Wed, 30 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-195-knn-implementation/https://wzk.icu/en/blog/bigdata-195-knn-implementation/KNN/K-Nearest Neighbors Algorithm: From Euclidean distance calculation, distance sorting, TopK voting to function encapsulation, giving reproducible Python.Tue, 29 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-196-sklearn-knn/https://wzk.icu/en/blog/bigdata-196-sklearn-knn/Since being initiated in 2007 by David Cournapeau, scikit-learn (sklearn) has become one of the most important machine learning libraries in the Python ecosystem.Tue, 29 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-193-apache-tez/https://wzk.icu/en/blog/bigdata-193-apache-tez/Tez (pronounced "tez") is an efficient data processing framework running in the Hadoop ecosystem, designed to optimize batch processing and interactive queries.Mon, 28 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-194-ml-overview/https://wzk.icu/en/blog/bigdata-194-ml-overview/In a bar, there are ten almost identical glasses of wine on the counter. The boss says want to play a game, win and drink for free, lose and pay three times for the wine.Mon, 28 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-191-elasticsearch-cluster-planning/https://wzk.icu/en/blog/bigdata-191-elasticsearch-cluster-planning/Master / Data / Coordinating node responsibilities and production role isolation strategies, capacity planning calculations.Sat, 26 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-192-datax-architecture/https://wzk.icu/en/blog/bigdata-192-datax-architecture/Scenario: Offline sync MySQL/HDFS/Hive/OTS/ODPS and other heterogeneous data sources, batch migration and data warehouse ETL.Sat, 26 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-75-spark-super-wordcount/https://wzk.icu/en/blog/bigdata-75-spark-super-wordcount/This is article 75 in the Big Data series, on top of basic WordCount add text preprocessing and database persistence, build a near-production word frequency pipeline.Sat, 26 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-76-spark-serialization/https://wzk.icu/en/blog/bigdata-76-spark-serialization/This is article 76 in the Big Data series, systematically reviewing Spark process communication mechanism, serialization strategy and RDD execution principle.Sat, 26 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-189-elk-nginx/https://wzk.icu/en/blog/bigdata-189-elk-nginx/Configure Nginx logformat json to output structured accesslog (containing @timestamp, requesttime, status, requesturi, ua and other fields).Fri, 25 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-190-filebeat-kafka-es/https://wzk.icu/en/blog/bigdata-190-filebeat-kafka-es/Filebeat collects Nginx access.log to Kafka, and Logstash consumes, parses embedded JSON by field conditions, enriches metadata, and writes structured logs to Elasticsearch for Kibana analysis.Fri, 25 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-187-logstash-filter/https://wzk.icu/en/blog/bigdata-187-logstash-filter/Filter is responsible for parsing, transforming, filtering events. Multiple Filters execute in configured order.Thu, 24 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-188-logstash-output/https://wzk.icu/en/blog/bigdata-188-logstash-output/Output is the final stage of Logstash pipeline, responsible for outputting processed data to target system.Thu, 24 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-185-logstash-intro/https://wzk.icu/en/blog/bigdata-185-logstash-intro/Logstash 7 getting started tutorial, covering stdin/file collection, sincedb mechanism and start_position effect conditions, with error quick reference tableWed, 23 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-186-logstash-input/https://wzk.icu/en/blog/bigdata-186-logstash-input/Logstash Input plugin comparison, breakdown technical differences between JDBC Input and Syslog collection pipeline, applicable scenarios and key configs.Wed, 23 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-73-spark-wordcount/https://wzk.icu/en/blog/bigdata-73-spark-wordcount/Implement distributed WordCount using Spark + Scala and Spark + Java, detailed RDD five-step processing flow, Maven project configuration and spark-submit command.Wed, 23 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-74-spark-scala-cases/https://wzk.icu/en/blog/bigdata-74-spark-scala-cases/Deep dive into Spark RDD programming through two classic cases: Monte Carlo method distributed Pi estimation, and mutual friends analysis in social networks with two approaches, comparing Cartesian.Wed, 23 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-183-elasticsearch-concurrency/https://wzk.icu/en/blog/bigdata-183-elasticsearch-concurrency/Elasticsearch concurrency conflicts (inventory deduction read-modify-write) breakdown write overwrite cause, and gives engineering solution using ES optimistic.Tue, 22 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-184-elasticsearch-doc-values/https://wzk.icu/en/blog/bigdata-184-elasticsearch-doc-values/Disk columnar data structure generated at indexing time, optimized for sorting, aggregation and script valuesTue, 22 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-181-elasticsearch-segment-merge/https://wzk.icu/en/blog/bigdata-181-elasticsearch-segment-merge/Explains why refresh causes small segment increase, how segment merge merges small segments into large ones in background and cleans deleted documents.Mon, 21 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-182-elasticsearch-inverted-index/https://wzk.icu/en/blog/bigdata-182-elasticsearch-inverted-index/Article details core data structure of Elasticsearch inverted index: Terms Dictionary, Posting List, FST (Finite State Transducer) and SkipList how accelerate.Mon, 21 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-179-elasticsearch-inverted-index/https://wzk.icu/en/blog/bigdata-179-elasticsearch-inverted-index/This article deeply analyzes Elasticsearch's inverted index principle based on Lucene, and document read/write flow.Sun, 20 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-180-elasticsearch-nrt/https://wzk.icu/en/blog/bigdata-180-elasticsearch-nrt/Article details core mechanism of Elasticsearch near real-time search, including Lucene Segment, Memory Buffer, File System Cache, Refresh, Flush and Translog.Sun, 20 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-72-spark-action/https://wzk.icu/en/blog/bigdata-72-spark-action/This is article 72 in the Big Data series, systematically reviewing Spark RDD Action operators.Sat, 19 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-177-elasticsearch-aggregation/https://wzk.icu/en/blog/bigdata-177-elasticsearch-aggregation/Covers complete practice of Metrics Aggregations and Bucket Aggregations, applicable to common Elasticsearch 7.x / 8.x versions in 2025.Fri, 18 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-178-elasticsearch-java/https://wzk.icu/en/blog/bigdata-178-elasticsearch-java/This article details the complete flow for index and document CRUD operations using Elasticsearch 7.3.0 and RestHighLevelClient.Fri, 18 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-175-elasticsearch-term-bool/https://wzk.icu/en/blog/bigdata-175-elasticsearch-term-bool/This article demonstrates Elasticsearch term-level queries including term, terms, range, exists, prefix, regexp, fuzzy, ids queries, and bool compound queries.Thu, 17 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-176-elasticsearch-filter-dsl/https://wzk.icu/en/blog/bigdata-176-elasticsearch-filter-dsl/This article details practical usage of Elasticsearch Filter DSL, covering filter query, sort pagination, highlight display and batch operations.Thu, 17 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-173-elasticsearch-mapping-crud/https://wzk.icu/en/blog/bigdata-173-elasticsearch-mapping-crud/After creating an index, need to set field constraints, called field mapping (mapping).Wed, 16 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-174-elasticsearch-query-dsl/https://wzk.icu/en/blog/bigdata-174-elasticsearch-query-dsl/In-depth explanation of core Query DSL usage in Elasticsearch 7.3, focusing on differences and pitfalls of match, matchphrase, querystring.Wed, 16 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-71-spark-cluster-arch/https://wzk.icu/en/blog/bigdata-71-spark-cluster-arch/This is article 71 in the Big Data series, introducing Spark cluster core architecture, deployment mode comparisons, and static/dynamic resource management strategies.Wed, 16 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-171-elasticsearch-head-kibana/https://wzk.icu/en/blog/bigdata-171-elasticsearch-head-kibana/Introduction to Elasticsearch-Head plugin and Kibana 7.3.0 installation and connectivity points, covering Chrome extension quick access.Tue, 15 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-172-elasticsearch-ik-analyzer/https://wzk.icu/en/blog/bigdata-172-elasticsearch-ik-analyzer/This article explains Elasticsearch index CRUD operations and IK analyzer config, covering versions 7.3.0 and 8.15.0.Tue, 15 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-169-elasticsearch-ru-men-dao-ke-yong/https://wzk.icu/en/blog/bigdata-169-elasticsearch-ru-men-dao-ke-yong/Elasticsearch (ES 7.x/8.x) minimum examples for index creation, document CRUD, query by ID, and _search, with response samples and screenshots to quickly run through the full index/document workflow.Mon, 14 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-170-elasticsearch-san-jie-dian-ji-qun-ying-zhan/https://wzk.icu/en/blog/bigdata-170-elasticsearch-san-jie-dian-ji-qun-ying-zhan/Elasticsearch 7.3.0 three-node cluster deployment practice tutorial, covering directory creation and permission settings.Mon, 14 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-167-elk-elastic-stack-ying-zhan/https://wzk.icu/en/blog/bigdata-167-elk-elastic-stack-ying-zhan/Article introduces core capabilities and common practices of Elasticsearch 8.x, Logstash 8.x, Kibana 8.Sun, 13 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-168-elasticsearch-dan-ji-bu-shu-yun-xing/https://wzk.icu/en/blog/bigdata-168-elasticsearch-dan-ji-bu-shu-yun-xing/Elasticsearch is a distributed full-text search engine, supports single-node mode and cluster mode deployment. Generally, small companies can use Single-Node Mode for their business scenarios.Sun, 13 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-165-apache-kylin-cube7-ying-zhan/https://wzk.icu/en/blog/bigdata-165-apache-kylin-cube7-ying-zhan/Covers Aggregation Group, Mandatory Dimension, Hierarchy Dimension, Joint Dimension usage trade-offs, and explains impact of dictionary encoding, RowKey order.Sat, 12 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-166-apache-kylin-streaming-cubing-ying-zhan/https://wzk.icu/en/blog/bigdata-166-apache-kylin-streaming-cubing-ying-zhan/Kafka→Kylin real-time OLAP pipeline, providing minute-level aggregation queries for common 2025 business scenarios (e-commerce transactions, user behavior, IoT monitoring).Sat, 12 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-69-spark-rdd/https://wzk.icu/en/blog/bigdata-69-spark-rdd/This is article 69 in the Big Data series, deeply analyzing RDD, Spark's core data abstraction, its five key features and design principles.Sat, 12 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-70-spark-rdd-operations/https://wzk.icu/en/blog/bigdata-70-spark-rdd-operations/This is article 70 in the Big Data series, comprehensively explaining Spark RDD's three creation methods and practical usage of common Transformation operators.Sat, 12 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-163-apache-kylin-segment-he-bing-ying-zhan/https://wzk.icu/en/blog/bigdata-163-apache-kylin-segment-he-bing-ying-zhan/Apache Kylin Segment merge practice tutorial, covering manual MERGE Job flow, continuous Segment requirements, Auto Merge multi-level threshold strategy, Retention Threshold cleanup logic.Fri, 11 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-164-apache-kylin-cuboid-jian-ying-zhan/https://wzk.icu/en/blog/bigdata-164-apache-kylin-cuboid-jian-ying-zhan/Cuboid pruning optimization: When there are many dimensions, Cuboid count grows exponentially, causing long build time and storage expansion.Fri, 11 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-161-apache-kylin-cube-ying-zhan/https://wzk.icu/en/blog/bigdata-161-apache-kylin-cube-ying-zhan/Apache Kylin 4.0 Cube modeling and query acceleration method: Complete star modeling with fact tables and dimension tables, design dimensions and measures.Thu, 10 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-162-apache-kylin-zeng-liang-cube-segment-ying-zhan/https://wzk.icu/en/blog/bigdata-162-apache-kylin-zeng-liang-cube-segment-ying-zhan/Using date field of Hive partitioned table as Partition Date Column, split Cube into multiple Segments, incrementally build by range to avoid repeated computation of historical dataThu, 10 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-159-apache-kylin-cube-1/https://wzk.icu/en/blog/bigdata-159-apache-kylin-cube-1/Apache Kylin is an open-source distributed analysis engine, focused on providing real-time OLAP (Online Analytical Processing) capabilities for big data.Wed, 09 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-160-apache-kylin-cube-2/https://wzk.icu/en/blog/bigdata-160-apache-kylin-cube-2/Scenario: Using e-commerce sales fact table, pre-compute aggregation queries accelerated by "date" dimension on Kylin.Wed, 09 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-67-spark-intro/https://wzk.icu/en/blog/bigdata-67-spark-intro/Systematic overview of big data processing engine evolution from MapReduce to Spark to Flink, analyzing Spark in-memory computing model, unified ecosystem and core components.Wed, 09 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-157-apache-kylin-intro/https://wzk.icu/en/blog/bigdata-157-apache-kylin-intro/Background, evolution and engineering practice of Apache Kylin, focusing on MOLAP solution implementation path for massive data analysis.Tue, 08 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-158-apache-kylin-deployment/https://wzk.icu/en/blog/bigdata-158-apache-kylin-deployment/Complete deployment record of Apache Kylin 3.1.1 on Hadoop 2.9.2, Hive 2.3.9, HBase 1.3.1, Spark 2.4.5 (without-hadoop.Tue, 08 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-65-kafka-storage/https://wzk.icu/en/blog/bigdata-65-kafka-storage/This is article 65 in the Big Data series, deeply analyzing Kafka's log storage mechanism.Sat, 05 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-66-kafka-highperformance/https://wzk.icu/en/blog/bigdata-66-kafka-highperformance/This is article 66 in the Big Data series, deeply analyzing Kafka's underlying I/O optimization technologies achieving extremely high throughput.Sat, 05 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-63-kafka-replica/https://wzk.icu/en/blog/bigdata-63-kafka-replica/Deep dive into Kafka replica mechanism, including ISR sync node set maintenance, Leader election process, and unclean election trade-offs between consistency and availability.Wed, 02 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-64-kafka-exactly-once/https://wzk.icu/en/blog/bigdata-64-kafka-exactly-once/Systematic explanation of how Kafka achieves Exactly-Once semantics through idempotent producers and transactions, covering PID/sequence number principle, cross-partition transaction configuration.Wed, 02 Oct 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-155-apache-druid-storage-query/https://wzk.icu/en/blog/bigdata-155-apache-druid-storage-query/Apache Druid data storage and high-performance query path: from DataSource/Chunk/Segment layering, to columnar storage, Roll-up pre-aggregation, Bitmap.Mon, 30 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-156-apache-druid-kafka-realtime/https://wzk.icu/en/blog/bigdata-156-apache-druid-kafka-realtime/Scala Kafka Producer writes order/click data to Kafka Topic (example topic: druid2), continuous ingestion in Druid through Kafka Indexing Service.Mon, 30 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-153-apache-druid-kafka-ingestion/https://wzk.icu/en/blog/bigdata-153-apache-druid-kafka-ingestion/Complete practice of Apache Druid real-time Kafka ingestion, using network traffic JSON as example, completing data ingestion through Druid console's Streaming/Kafka wizard, parsing time column.Sun, 29 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-154-apache-druid-architecture/https://wzk.icu/en/blog/bigdata-154-apache-druid-architecture/Apache Druid component responsibilities and deployment points from 0.13.0 to current (2025): Coordinator manages Historical node Segment.Sun, 29 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-151-apache-druid-cluster-1/https://wzk.icu/en/blog/bigdata-151-apache-druid-cluster-1/Scenario: 2C4G/2C2G three-node mixed deployment, Druid 30.0.0, Kafka/HDFS/MySQL collaboration. Conclusion: Can run on low config, but core is DirectMemory and processing.Sat, 28 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-152-apache-druid-cluster-2/https://wzk.icu/en/blog/bigdata-152-apache-druid-cluster-2/Low-memory cluster practice for Apache Druid 30.0.0 on three nodes: provides JVM parameters and runtime.Sat, 28 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-61-kafka-topic-partition/https://wzk.icu/en/blog/bigdata-61-kafka-topic-partition/Deep dive into Kafka Topic, Partition, Consumer Group core mechanisms, covering custom deserialization, offset management and rebalance optimization configuration.Sat, 28 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-62-kafka-topic-management/https://wzk.icu/en/blog/bigdata-62-kafka-topic-management/Comprehensive introduction to Kafka Topic operations, including kafka-topics.sh commands, replica assignment strategy principles, and KafkaAdminClient Java API core usage.Sat, 28 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-149-apache-druid-architecture/https://wzk.icu/en/blog/bigdata-149-apache-druid-architecture/Apache Druid real-time OLAP practice: suitable for event detail with time as primary key, sub-second aggregation and high-concurrency self-service analysis.Fri, 27 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-150-apache-druid-deployment/https://wzk.icu/en/blog/bigdata-150-apache-druid-deployment/Scenario: Quickly experience Apache Druid 30.0.0 locally/single-machine, verify real-time and historical queries and console access.Fri, 27 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-148-flink-kudu-sink/https://wzk.icu/en/blog/bigdata-148-flink-kudu-sink/Complete runnable example for Kudu, based on Flink 1.11.1 (Scala 2.12)/Java 11 and kudu-client 1.17.0 (2025 test).Wed, 25 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-59-kafka-interceptor/https://wzk.icu/en/blog/bigdata-59-kafka-interceptor/Introduction to Kafka 0.10 Producer interceptor mechanism, covering onSend and onAcknowledgement interception points, interceptor chain execution order and error isolation.Wed, 25 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-60-kafka-consumer/https://wzk.icu/en/blog/bigdata-60-kafka-consumer/Detailed explanation of Kafka Consumer Group consumption model, partition assignment strategy, heartbeat keep-alive mechanism, and tuning practices for key parameters like session.timeout.Wed, 25 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-146-apache-kudu-docker-deployment/https://wzk.icu/en/blog/bigdata-146-apache-kudu-docker-deployment/Apache Kudu Docker Compose quick deployment solution on Ubuntu 22.04 cloud host, covering Kudu Master and Tablet Server components.Tue, 24 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-147-java-kudu-crud/https://wzk.icu/en/blog/bigdata-147-java-kudu-crud/Java client (kudu-client 1.4.0) connects to Apache Kudu with multiple Masters (example ports 7051/7151/7251), completes full process of table creation.Tue, 24 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-144-apache-kudu-intro/https://wzk.icu/en/blog/bigdata-144-apache-kudu-intro/Apache Kudu is an open-source storage engine developed by Cloudera and contributed to Apache Software Foundation.Mon, 23 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-145-apache-kudu-architecture/https://wzk.icu/en/blog/bigdata-145-apache-kudu-architecture/Apache Kudu's Master/TabletServer architecture, RowSet (MemRowSet/DiskRowSet) write/read path, MVCC, and Raft consensus role in replica and failoverMon, 23 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-143-clickhouse-mergetree-partition-ttl/https://wzk.icu/en/blog/bigdata-143-clickhouse-mergetree-partition-ttl/ClickHouse is a columnar database for OLAP (Online Analytical Processing), favored in big data analysis for its high-speed data processing.Sat, 21 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-57-kafka-producer-advanced/https://wzk.icu/en/blog/bigdata-57-kafka-producer-advanced/Deep analysis of Kafka Producer initialization, message interception, serialization, partition routing, buffer batch sending, ACK confirmation and complete sending chain.Sat, 21 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-58-kafka-serialization-partition/https://wzk.icu/en/blog/bigdata-58-kafka-serialization-partition/Deep dive into Kafka message serialization and partition routing, including complete code for custom Serializer and Partitioner, mastering precise message routing and efficient transmission.Sat, 21 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-141-clickhouse-replicated-mergetree-zookeeper/https://wzk.icu/en/blog/bigdata-141-clickhouse-replicated-mergetree-zookeeper/ReplicatedMergeTree ZooKeeper: Implements communication between multiple instances.Fri, 20 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-142-clickhouse-shard-replica-distributed/https://wzk.icu/en/blog/bigdata-142-clickhouse-shard-replica-distributed/Replica refers to storing the same data on different physical nodes in a distributed system. Its core idea is to improve system reliability through data redundancy.Fri, 20 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-139-clickhouse-mergetree-best-practice/https://wzk.icu/en/blog/bigdata-139-clickhouse-mergetree-best-practice/Scenario: Solve two common "quasi-real-time detail table" requirements: deduplication/update and key-based summing. Conclusion: ReplacingMergeTree for deduplication by sort key / latest version by versionThu, 19 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-140-clickhouse-collapsing-mergetree/https://wzk.icu/en/blog/bigdata-140-clickhouse-collapsing-mergetree/ClickHouse external data source engine guide: DDL templates, key parameters and read/write pipelines for ENGINE=HDFS, ENGINE=MySQL, ENGINE=Kafka, and distributed table configurations.Thu, 19 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-137-clickhouse-mergetree-guide/https://wzk.icu/en/blog/bigdata-137-clickhouse-mergetree-guide/ClickHouse MergeTree key mechanisms: batch writes form parts, background merge (Compact/Wide two part forms).Wed, 18 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-138-clickhouse-mergetree-details/https://wzk.icu/en/blog/bigdata-138-clickhouse-mergetree-details/ClickHouse MergeTree storage and query path: column files (*.bin), sparse primary index (primary.idx), marker files (.mrk/.Wed, 18 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-55-kafka-operations/https://wzk.icu/en/blog/bigdata-55-kafka-operations/Covers Kafka daily operations: daemon startup, Shell topic management commands, and Java client programming (complete Producer/Consumer code) with key configuration parameters and ConsumerRebalance.Wed, 18 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-56-kafka-springboot/https://wzk.icu/en/blog/bigdata-56-kafka-springboot/Detailed guide on integrating Kafka in Spring Boot projects, including dependency configuration, KafkaTemplate sync/async message sending, and complete @KafkaListener consumption practice.Wed, 18 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-68-spark-install/https://wzk.icu/en/blog/bigdata-68-spark-install/Step-by-step Apache Spark distributed computing environment setup, covering download and extract, environment variable configuration, slaves/spark-env.Wed, 18 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-135-clickhouse-cluster-check-datatype/https://wzk.icu/en/blog/bigdata-135-clickhouse-cluster-check-datatype/Using three-node cluster (h121/122/123) as example, first complete cluster connectivity self-check: system.Sat, 14 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-136-clickhouse-table-engine/https://wzk.icu/en/blog/bigdata-136-clickhouse-table-engine/Scenario: Need to trade-off among small data/temporary table/log landing/multi-table combined reads, often using MergeTree is "using a cannon to kill a mosquito".Sat, 14 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-53-kafka-components/https://wzk.icu/en/blog/bigdata-53-kafka-components/Deep dive into Kafka's three core components: Producer partitioning strategy and ACK mechanism, Broker Leader/Follower architecture, Consumer Group partition assignment and offset management.Sat, 14 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-54-kafka-install/https://wzk.icu/en/blog/bigdata-54-kafka-install/Introduction to Kafka 2.x vs 3.x core differences, detailed cluster installation steps, ZooKeeper configuration, Broker parameter settings, and how KRaft mode replaces ZooKeeper dependency.Sat, 14 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-133-clickhouse-concept-basics/https://wzk.icu/en/blog/bigdata-133-clickhouse-concept-basics/Scenario: Want high-concurrency low-latency OLAP, and don't want to use entire Hadoop/lakehouse. Conclusion: ClickHouse's core lies in columnar+vectorized+MergeTree+approximate statisticsFri, 13 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-134-clickhouse-install-cluster/https://wzk.icu/en/blog/bigdata-134-clickhouse-install-cluster/Official recommended keyring + signed-by installation of ClickHouse on Ubuntu, start with systemd and self-checkFri, 13 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-131-flink-cep-24h-5times-10min-unpaid/https://wzk.icu/en/blog/bigdata-131-flink-cep-24h-5times-10min-unpaid/Flink CEP (Complex Event Processing) is an extension library provided by Apache Flink for real-time complex event processing. Through Flink CEP, developers can identify specific event patterns from stream data.Thu, 12 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-132-flink-sql-table-api/https://wzk.icu/en/blog/bigdata-132-flink-sql-table-api/Engineering perspective to quickly run Flink SQL: Provides modern dependencies (no longer using blink planner), minimum runnable example (MRE).Thu, 12 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-129-flink-cep/https://wzk.icu/en/blog/bigdata-129-flink-cep/Flink CEP (Complex Event Processing) is a core component of Apache Flink, specifically designed for processing complex event streams.Wed, 11 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-130-flink-cep-timeout/https://wzk.icu/en/blog/bigdata-130-flink-cep-timeout/Flink CEP timeout event extraction is a key step in stream processing, used to capture partial matching events that exceed the window time (within) during pattern matching, enabling timeout order detection.Wed, 11 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-51-redis-sentinel/https://wzk.icu/en/blog/bigdata-51-redis-sentinel/This is article 51 in the Big Data series, covering Redis high availability architecture: master-slave replication, Sentinel mode, and distributed lock design.Wed, 11 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-52-kafka-architecture/https://wzk.icu/en/blog/bigdata-52-kafka-architecture/Systematic introduction to Kafka core architecture: Topic/Partition/Replica model, ISR mechanism, zero-copy optimization, message format and typical use cases.Wed, 11 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-127-flink-statebackend-detailed/https://wzk.icu/en/blog/bigdata-127-flink-statebackend-detailed/ManagedOperatorState is used to manage non-keyed state, achieving state consistency when operators recover from faults or scale.Tue, 10 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-128-flink-parallelism-detail/https://wzk.icu/en/blog/bigdata-128-flink-parallelism-detail/Basic Concept of Parallelism In Apache Flink, Parallelism refers to the number of parallel tasks that can run simultaneously for each operator during execution.Tue, 10 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-125-flink-broadcast-state/https://wzk.icu/en/blog/bigdata-125-flink-broadcast-state/Broadcast State is an important mechanism in Apache Flink that supports dynamic logic updates in streaming applications.Mon, 09 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-126-flink-state-backend/https://wzk.icu/en/blog/bigdata-126-flink-state-backend/State Storage Methods: MemoryStateBackend: Stores state in TaskManager's Java memory. Fast but limited (5MB per state default, 10MB per task).Mon, 09 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-123-flink-parallelism/https://wzk.icu/en/blog/bigdata-123-flink-parallelism/A Flink program consists of multiple Operators (Source, Transformation, Sink).Sat, 07 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-124-flink-state/https://wzk.icu/en/blog/bigdata-124-flink-state/Based on whether intermediate state is needed, Flink computation can be divided into stateful and stateless: Stateless computation like Map, Filter, FlatMap.Sat, 07 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-49-redis-cache-problems/https://wzk.icu/en/blog/bigdata-49-redis-cache-problems/Systematic overview of the five most common Redis cache problems in high-concurrency scenarios: cache penetration, cache breakdown, cache avalanche, hot key, and big key.Sat, 07 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-50-redis-distributed-lock/https://wzk.icu/en/blog/bigdata-50-redis-distributed-lock/Redis optimistic lock in practice: WATCH/MULTI/EXEC mechanism explained, Lua scripts for atomic operations, SETNX+EXPIRE distributed lock from basics to Redisson, with complete Java code examples.Sat, 07 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-121-flink-time-semantic/https://wzk.icu/en/blog/bigdata-121-flink-time-semantic/Watermark is a special marker used to tell Flink the progress of events in the data stream.Fri, 06 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-122-flink-watermark/https://wzk.icu/en/blog/bigdata-122-flink-watermark/When using event-time based windows, Flink relies on Watermark to decide when to trigger window computation.Fri, 06 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-119-flink-window-analysis/https://wzk.icu/en/blog/bigdata-119-flink-window-analysis/Flink's Window mechanism is the core bridge between stream processing and unified batch processing architecture.Thu, 05 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-120-flink-sliding-window/https://wzk.icu/en/blog/bigdata-120-flink-sliding-window/Sliding window is a more generalized form of fixed window, achieving dynamic window movement through introducing slide interval. It consists of two key parametersThu, 05 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-117-flink-jdbc-sink/https://wzk.icu/en/blog/bigdata-117-flink-jdbc-sink/In Apache Flink, JDBC Sink is an important data output component that allows writing stream or batch processed data to relational databases through JDBC connections.Wed, 04 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-118-flink-dataset-api/https://wzk.icu/en/blog/bigdata-118-flink-dataset-api/Apache Flink's DataSet API is the core programming interface for Flink batch processing, specifically designed for processing static, bounded datasets.Wed, 04 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-47-redis-memory-eviction/https://wzk.icu/en/blog/bigdata-47-redis-memory-eviction/Comprehensive analysis of Redis memory control mechanisms, including maxmemory configuration, three key expiration deletion strategies (lazy/active/scheduled).Wed, 04 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-48-redis-resp-reactor/https://wzk.icu/en/blog/bigdata-48-redis-resp-reactor/This is article 48 in the Big Data series. This article provides an in-depth analysis of Redis communication protocol RESP and Reactor-based event-driven architecture.Wed, 04 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-115-flink-datastream-transformation/https://wzk.icu/en/blog/bigdata-115-flink-datastream-transformation/Flink provides rich operators for DataStream to support flexible data stream processing in different scenarios.Tue, 03 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-116-flink-sink-usage/https://wzk.icu/en/blog/bigdata-116-flink-sink-usage/Flink's Sink is the final output endpoint for data stream processing, used to write processed results to external systems or storage media.Tue, 03 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-113-flink-non-parallel-source/https://wzk.icu/en/blog/bigdata-113-flink-non-parallel-source/Non-Parallel Source is a source operation in Flink with fixed parallelism of 1. It can only run in a single instance regardless of cluster scale, ensuring tasks are processed sequentially.Mon, 02 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-114-flink-rich-source-function/https://wzk.icu/en/blog/bigdata-114-flink-rich-source-function/RichSourceFunction and RichParallelSourceFunction are enhanced source functions suitable for scenarios requiring complex logic and resource management.Mon, 02 Sep 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-111-flink-yarn-deployment/https://wzk.icu/en/blog/bigdata-111-flink-yarn-deployment/Deploying Flink in YARN mode requires completing a series of environment configuration and cluster management operations.Sat, 31 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-112-flink-datastream-api/https://wzk.icu/en/blog/bigdata-112-flink-datastream-api/DataSource, Transformation and Sink. DataSource provides diverse data input methods including file systems, message queues, databases and custom data sources.Sat, 31 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-45-redis-persistence/https://wzk.icu/en/blog/bigdata-45-redis-persistence/Systematic comparison of Redis two persistence solutions: RDB snapshot and AOF log — configuration methods, trigger mechanisms, pros and cons, AOF rewrite mechanism.Sat, 31 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-46-redis-rdb/https://wzk.icu/en/blog/bigdata-46-redis-rdb/In-depth analysis of Redis RDB persistence mechanism, covering trigger methods, BGSAVE execution flow, configuration parameters, file structure, and comparison with AOF.Sat, 31 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-109-flink-architecture/https://wzk.icu/en/blog/bigdata-109-flink-architecture/Flink's runtime architecture adopts typical Master/Slave pattern with clear division of responsibilities among core components.Fri, 30 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-110-flink-install/https://wzk.icu/en/blog/bigdata-110-flink-install/Flink provides multiple installation modes to suit different scenarios.Fri, 30 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-107-flink-intro/https://wzk.icu/en/blog/bigdata-107-flink-intro/Apache Flink is an open-source big data stream processing framework, supporting efficient computation of unbounded stream and bounded batch data.Thu, 29 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-108-flink-wordcount/https://wzk.icu/en/blog/bigdata-108-flink-wordcount/Definition: Stream processing means real-time processing of continuously flowing data streams.Thu, 29 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-43-redis-lua/https://wzk.icu/en/blog/bigdata-43-redis-lua/Systematic explanation of Redis Lua script EVAL command syntax, differences between redis.call and redis.Wed, 28 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-44-redis-slowlog/https://wzk.icu/en/blog/bigdata-44-redis-slowlog/Detailed explanation of Redis slow query log configuration parameters (slowlog-log-slower-than, slowlog-max-len), core commands.Wed, 28 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-103-spark-streaming-kafka-offset/https://wzk.icu/en/blog/bigdata-103-spark-streaming-kafka-offset/When Spark Streaming integrates with Kafka, Offset management is key to ensuring data processing continuity and consistency.Tue, 27 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-104-spark-streaming-kafka-offset-redis/https://wzk.icu/en/blog/bigdata-104-spark-streaming-kafka-offset-redis/Offset is used to mark message position in Kafka partition. Proper management can achieve at-least-once or even exactly-once data processing semantics.Tue, 27 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-101-spark-streaming-window/https://wzk.icu/en/blog/bigdata-101-spark-streaming-window/Window operations integrate data from multiple batches over a longer time range by setting window length and slide duration.Mon, 26 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-102-spark-streaming-kafka/https://wzk.icu/en/blog/bigdata-102-spark-streaming-kafka/This article introduces two Spark Streaming integration methods with Kafka: Receiver Approach and Direct Approach.Mon, 26 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-41-redis-advanced-types/https://wzk.icu/en/blog/bigdata-41-redis-advanced-types/Deep dive into Redis three advanced data types: Bitmap, Geo (GeoHash, Z-order curve, Base32 encoding), and Stream message stream, with common commands and practical examples.Sat, 24 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-42-redis-pubsub/https://wzk.icu/en/blog/bigdata-42-redis-pubsub/Detailed explanation of Redis Pub/Sub working mechanism, three weak transaction flaws (no persistence, no acknowledgment, no retry), and alternative solutions in production.Sat, 24 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-39-redis-install/https://wzk.icu/en/blog/bigdata-39-redis-install/Install Redis 6.2.9 from source on Ubuntu, configure redis.conf for daemon mode, start redis-server and verify connection via redis-cli.Wed, 21 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-40-redis-data-types/https://wzk.icu/en/blog/bigdata-40-redis-data-types/Comprehensive explanation of Redis five data types: String, List, Set, Sorted Set, and Hash. Includes common commands, underlying characteristics, and typical usage scenarios with practical examples.Wed, 21 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-37-hbase-java-api/https://wzk.icu/en/blog/bigdata-37-hbase-java-api/Using HBase Java Client API to implement table creation, insert, delete, Get query, full table scan, and range scan.Sat, 17 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-38-redis-intro/https://wzk.icu/en/blog/bigdata-38-redis-intro/Introduction to Redis: in-memory data structure store, key-value database, with comparison to traditional databases and typical use cases.Sat, 17 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-35-hbase-cluster/https://wzk.icu/en/blog/bigdata-35-hbase-cluster/This is article 35 in the Big Data series. Complete HBase distributed cluster deployment on three-node Hadoop + ZooKeeper cluster.Wed, 14 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-36-hbase-crud/https://wzk.icu/en/blog/bigdata-36-hbase-crud/HBase Shell commands: create table, Put/Get/Scan/Delete operations, explain HBase data model with practical examples.Wed, 14 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-33-hbase-architecture/https://wzk.icu/en/blog/bigdata-33-hbase-architecture/Comprehensive analysis of HBase distributed database overall architecture, including ZooKeeper coordination, HMaster management node, HRegionServer data node, Region storage unit.Sat, 10 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-34-hbase-single-node/https://wzk.icu/en/blog/bigdata-34-hbase-single-node/Step-by-step configure HBase single node environment, explain hbase-env.sh, hbase-site.xml key parameters, complete integration with Hadoop HDFS and ZooKeeper cluster.Sat, 10 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-31-zookeeper-leader-election/https://wzk.icu/en/blog/bigdata-31-zookeeper-leader-election/This is article 31 in the Big Data series. Deep analysis of ZooKeeper Leader election mechanism and ZAB (ZooKeeper Atomic Broadcast) protocol implementation principles.Wed, 07 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-32-zookeeper-distributed-lock/https://wzk.icu/en/blog/bigdata-32-zookeeper-distributed-lock/This is article 32 in the Big Data series. Demonstrates how to implement fair distributed lock using ZooKeeper ephemeral sequential nodes, with complete Java code.Wed, 07 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-29-zookeeper-watcher-practice/https://wzk.icu/en/blog/bigdata-29-zookeeper-watcher-practice/Complete analysis of Watcher registration-trigger-notification flow from client, WatchManager to ZooKeeper server, and zkCli command line practice demonstrating node CRUD and monitoring.Sat, 03 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-30-zookeeper-java-api/https://wzk.icu/en/blog/bigdata-30-zookeeper-java-api/Use ZkClient library to operate ZooKeeper via Java code, complete practical examples of session establishment, persistent node CRUD, child node change monitoring, and data change monitoring.Sat, 03 Aug 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-27-zookeeper-config/https://wzk.icu/en/blog/bigdata-27-zookeeper-config/Deep dive into zoo.cfg core parameter meanings, explain myid file configuration specifications, demonstrate 3-node cluster startup process and Leader election result verification.Wed, 31 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-28-zookeeper-znode-watcher/https://wzk.icu/en/blog/bigdata-28-zookeeper-znode-watcher/Deep dive into ZooKeeper's four ZNode node types, ZXID transaction ID structure, and one-time trigger Watcher monitoring mechanism principles and practice.Wed, 31 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-25-sqoop-cdc/https://wzk.icu/en/blog/bigdata-25-sqoop-cdc/Introduce Sqoop's --incremental append incremental import mechanism, and deeply explain CDC (Change Data Capture) core concepts, capture method comparisons, and modern solutions like Flink CDC, Deb.Sat, 27 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-26-zookeeper-intro/https://wzk.icu/en/blog/bigdata-26-zookeeper-intro/Introduction to ZooKeeper core concepts, Leader/Follower/Observer role division, ZAB protocol principles, and demonstration of 3-node cluster installation and configuration process.Sat, 27 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-23-sqoop-partial-import/https://wzk.icu/en/blog/bigdata-23-sqoop-partial-import/Detailed explanation of three ways Sqoop imports partial data from MySQL to HDFS by condition: custom query, specify columns, WHERE condition filtering, with applicable scenarios and precautions.Wed, 24 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-24-sqoop-hive/https://wzk.icu/en/blog/bigdata-24-sqoop-hive/Demonstrates Sqoop importing MySQL data directly to Hive table, and exporting Hive data back to MySQL, covering key parameters like --hive-import, --create-hive-table usage.Wed, 24 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-21-sqoop-intro/https://wzk.icu/en/blog/bigdata-21-sqoop-intro/Introduction to Apache Sqoop core principles, use cases, and installation configuration steps on Hadoop cluster, helping quickly get started with batch data migration between MySQL and HDFS/Hive.Sat, 20 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-22-sqoop-mysql-hdfs/https://wzk.icu/en/blog/bigdata-22-sqoop-mysql-hdfs/Complete example demonstrating Sqoop importing MySQL table data to HDFS, covering core parameter explanations, MapReduce parallel mechanism, and execution result verification.Sat, 20 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-19-flume-hdfs/https://wzk.icu/en/blog/bigdata-19-flume-hdfs/Use Flume exec source to real-time track Hive log files, buffer via memory channel, configure HDFS sink to write with time-based partitioning, implement automatic log data landing to HDFS.Wed, 17 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-20-flume-dual-sink/https://wzk.icu/en/blog/bigdata-20-flume-dual-sink/This is article 20 in the Big Data series. Demonstrates Flume replication mode with dual Sink architecture—same data written to both HDFS and local filesystem.Wed, 17 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-17-flume-intro/https://wzk.icu/en/blog/bigdata-17-flume-intro/Introduction to Apache Flume positioning, core components (Source, Channel, Sink), event model and common data flow topologies, and installation configuration methods.Sat, 13 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-18-flume-helloworld/https://wzk.icu/en/blog/bigdata-18-flume-helloworld/Through Flume's simplest Hello World case, use netcat source to monitor port, memory channel for buffering, logger sink for console output, demonstrating complete Source→Channel→Sink data flow.Sat, 13 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-15-hive-metastore/https://wzk.icu/en/blog/bigdata-15-hive-metastore/Detailed explanation of Hive Metastore's embedded, local, and remote deployment modes, and complete steps to configure high-availability remote Metastore on three-node cluster.Wed, 10 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-16-hive-hiveserver2/https://wzk.icu/en/blog/bigdata-16-hive-hiveserver2/Introduction to HiveServer2 architecture and role, configure Hadoop proxy user and WebHDFS, implement cross-node JDBC remote access to Hive via Beeline client.Wed, 10 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-13-hive-ddl-dml/https://wzk.icu/en/blog/bigdata-13-hive-ddl-dml/Systematic explanation of Hive DDL (database/table creation, internal and external tables) and DML (data loading, insertion, query) operations.Mon, 08 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-14-hive-hql-advanced/https://wzk.icu/en/blog/bigdata-14-hive-hql-advanced/Deep dive into Hive's multiple data import methods (LOAD/INSERT/External Table/Sqoop), data export methods, and practical usage of HQL query operations like aggregation, filtering, and sorting.Mon, 08 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-11-mapreduce-join/https://wzk.icu/en/blog/bigdata-11-mapreduce-join/This is article 11 in the Big Data series. Introduces four classic strategies for implementing multi-table JOIN in MapReduce framework and their Java implementations.Thu, 04 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-12-hive-intro/https://wzk.icu/en/blog/bigdata-12-hive-intro/Introduction to Hive data warehouse core concepts, architecture components and pros/cons, with detailed steps to install and configure Hive 2.3.9 on three-node Hadoop cluster.Thu, 04 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-09-hdfs-java-client/https://wzk.icu/en/blog/bigdata-09-hdfs-java-client/This is article 9 in the Big Data series. Learn to operate HDFS through Java code, master Hadoop's Java Client API.Wed, 03 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-10-mapreduce-java-wordcount/https://wzk.icu/en/blog/bigdata-10-mapreduce-java-wordcount/Implement Hadoop MapReduce WordCount from scratch: Hadoop serialization mechanism detailed explanation, writing Mapper, Reducer, Driver three components, Maven project configuration.Wed, 03 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-07-hdfs-read-write-principle/https://wzk.icu/en/blog/bigdata-07-hdfs-read-write-principle/Deep dive into HDFS architecture: NameNode, DataNode, Client roles, Block storage mechanism, file read/write process (Pipeline write and nearest read), and HDFS basic commands.Tue, 02 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-08-hdfs-cli-practice/https://wzk.icu/en/blog/bigdata-08-hdfs-cli-practice/Complete HDFS CLI practice: hadoop fs common commands including directory operations, file upload/download, permission management, with three-node cluster live demo.Tue, 02 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-05-hadoop-wordcount/https://wzk.icu/en/blog/bigdata-05-hadoop-wordcount/Complete WordCount execution on Hadoop cluster: upload files to HDFS, submit MapReduce job, view running status through YARN UI, verify true distributed computing.Mon, 01 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-06-hadoop-jobhistory/https://wzk.icu/en/blog/bigdata-06-hadoop-jobhistory/Configure Hadoop JobHistoryServer to record MapReduce job execution history, enable YARN log aggregation, view job details and logs via Web UI.Mon, 01 Jul 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-03-hadoop-ssh-passwordless/https://wzk.icu/en/blog/bigdata-03-hadoop-ssh-passwordless/Complete guide for Hadoop three-node cluster SSH passwordless login: generate RSA keys, distribute public keys, write rsync cluster distribution script.Sun, 30 Jun 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-04-hadoop-cluster-start/https://wzk.icu/en/blog/bigdata-04-hadoop-cluster-start/Complete startup process for Hadoop three-node cluster: format NameNode, start HDFS and YARN, verify cluster status via Web UI, including start-dfs.sh and start-yarn.sh usage.Sun, 30 Jun 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-01-hadoop-env/https://wzk.icu/en/blog/bigdata-01-hadoop-env/This article is migrated from Juejin. Original link: Big Data 01 - Basic Environment SetupFri, 28 Jun 2024 00:00:00 GMThttps://wzk.icu/en/blog/bigdata-02-hadoop-xml-config/https://wzk.icu/en/blog/bigdata-02-hadoop-xml-config/Detailed explanation of Hadoop cluster three-node XML configuration files: core-site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.Fri, 28 Jun 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-26-custom-agent/https://wzk.icu/en/blog/langchain-26-custom-agent/A Custom Agent refers to an intelligent agent program customized by users based on specific requirements, which can execute specific tasks or workflows.Mon, 15 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-24-agent-executor/https://wzk.icu/en/blog/langchain-24-agent-executor/This article introduces how to use the Langchain library in Python for document retrieval, load web content, configure OpenAIEmbeddings, and integrate GPT-3.5-turbo model for Q&A.Sun, 14 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-25-react-framework/https://wzk.icu/en/blog/langchain-25-react-framework/This article introduces ReAct, a framework that uses logical reasoning and action sequences to achieve goal-oriented tasks through LLM decision-making and operations.Sun, 14 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-22-text-embedding-faiss/https://wzk.icu/en/blog/langchain-22-text-embedding-faiss/Text Embedding involves the process of mapping high-dimensional data (such as text, images, etc.) to lower-dimensional spaces.Sat, 13 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-23-vector-ai-semantic-search/https://wzk.icu/en/blog/langchain-23-vector-ai-semantic-search/Vector Storage, also known as Vector Database, is a database system specifically optimized for storing and retrieving high-dimensional vector data.Sat, 13 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-20-document-loaders/https://wzk.icu/en/blog/langchain-20-document-loaders/This article introduces various document loaders provided by the LangChain library, such as TextLoader, CSVLoader, DirectoryLoader, etc., demonstrating how to load and process data in various formats.Fri, 12 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-21-text-splitter/https://wzk.icu/en/blog/langchain-21-text-splitter/This article introduces various TextSplitters in the LangChain library, including character-based, word-based, HTML tag-based, and programming language-based splitters, as well as their application.Fri, 12 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-18-cache/https://wzk.icu/en/blog/langchain-18-cache/LangChain provides a comprehensive caching mechanism to significantly reduce LLM call latency and costs. Its core includes InMemoryCache (in-memory cache) and SQLiteCache (persistent cache).Thu, 11 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-19-token-usage/https://wzk.icu/en/blog/langchain-19-token-usage/Explains how to integrate OpenAI GPT-3 model in Python through LangChain library, demonstrating how to use the getopenaicallback function to obtain callbacks and execute requests.Thu, 11 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-16-using-tools/https://wzk.icu/en/blog/langchain-16-using-tools/LangChain is a powerful open-source framework designed to help developers more efficiently build and deploy applications based on Large Language Models (LLMs).Wed, 10 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-17-function-calling/https://wzk.icu/en/blog/langchain-17-function-calling/Function Calling is a core technology for Large Language Models (like GPT-4, Claude, Gemini) to interact with external systems.Wed, 10 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-14-openai-moderation/https://wzk.icu/en/blog/langchain-14-openai-moderation/Moderation refers to the process of reviewing and managing user-generated content (UGC) through manual or automated means.Tue, 09 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-15-agent-executor/https://wzk.icu/en/blog/langchain-15-agent-executor/Build an intelligent knowledge retrieval system using Wikipedia search plugin, AgentExecutor, and LangChain tools. Covers agent initialization, tool binding, and multi-step reasoning workflows.Tue, 09 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-12-routing-by-semantic-similarity/https://wzk.icu/en/blog/langchain-12-routing-by-semantic-similarity/This article introduces a method using large models (like OpenAI) and Prompt templates to handle unexpected inputs in program design by calculating the similarity between queries and preset templates.Mon, 08 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-13-memory-conversation-buffer-memory/https://wzk.icu/en/blog/langchain-13-memory-conversation-buffer-memory/This article introduces how to use tools in the LangChain library to manage conversation context of large models in Python.Mon, 08 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-11-code-writing-function-calling/https://wzk.icu/en/blog/langchain-11-code-writing-function-calling/This article introduces how GPT models work based on autoregressive language modeling, which generates coherent text by predicting the probability of the next token.Sun, 07 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-09-query-sql-db-with-run-gpt/https://wzk.icu/en/blog/langchain-09-query-sql-db-with-run-gpt/RUN GPT provides a powerful database query function, allowing users to input natural language to query database content.Sat, 06 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-10-langchainhub/https://wzk.icu/en/blog/langchain-10-langchainhub/This article introduces how to use LangChainHub's Hub mechanism through Python code to easily access and share Prompts.Sat, 06 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-07-multiple-chains/https://wzk.icu/en/blog/langchain-07-multiple-chains/How to use Runnable and Prompts in LangChain to create chainable conversation flows for multi-stage question answering, with practical examples of sequential and parallel chain composition.Fri, 05 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-08-query-sql-db/https://wzk.icu/en/blog/langchain-08-query-sql-db/This article introduces how to use LangChain framework to import Chinook SQLite database through Python script and use GPT model to execute SQL queries, such as calculating employee count.Fri, 05 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-05-rag-conversational/https://wzk.icu/en/blog/langchain-05-rag-conversational/Conversational Search is an intelligent search technology that combines natural language processing and context understanding capabilities.Thu, 04 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-06-rag-with-source-document/https://wzk.icu/en/blog/langchain-06-rag-with-source-document/Retrieval-Augmented Generation (RAG) with Source Document is an AI technology framework that combines retrieval with large language model generation.Thu, 04 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-03-astream-events-faiss/https://wzk.icu/en/blog/langchain-03-astream-events-faiss/This article introduces how to use DocArrayInMemorySearch to vectorize text data, combined with OpenAIEmbeddings and GPT-3.5 model, to implement relevant information retrieval and answer generation.Wed, 03 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-04-rag-retrieval-augmented-generation/https://wzk.icu/en/blog/langchain-04-rag-retrieval-augmented-generation/This article explains in detail how to use RAG technology in LangChain, combined with OpenAI's GPT-3.5 model, to improve text generation quality through retrieval and generation.Wed, 03 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-01-hello-world/https://wzk.icu/en/blog/langchain-01-hello-world/This article introduces how to use the LangChain library with OpenAI API and GPT-3.5-turbo model to create a template for generating jokes about specific topics (like cats).Tue, 02 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/langchain-02-json-output-parser/https://wzk.icu/en/blog/langchain-02-json-output-parser/This article explains how to install and use LangChain and OpenAI API in Python, retrieve specified country and its population data through async functions.Tue, 02 Apr 2024 00:00:00 GMThttps://wzk.icu/en/blog/gitlab-jenkins-cicd/https://wzk.icu/en/blog/gitlab-jenkins-cicd/Learn how to set up a CI/CD pipeline using GitLab and Jenkins. Covers Webhook configuration, branch strategy, automated build scripts, and Docker image deployment to Kubernetes.Wed, 08 Feb 2023 00:00:00 GMThttps://wzk.icu/en/blog/docker-harbor-private-registry/https://wzk.icu/en/blog/docker-harbor-private-registry/Step-by-step guide to installing Docker and Harbor private registry, configuring insecure registries, building custom Docker images, and pushing them to Harbor.Thu, 02 Feb 2023 00:00:00 GMThttps://wzk.icu/en/blog/python-offline-deployment/https://wzk.icu/en/blog/python-offline-deployment/How to export Python packages in an online environment and install them in an air-gapped system using pip wheel, pip download, and requirements.txt for offline deployment.Tue, 17 Jan 2023 00:00:00 GMT