CLI-57: Add Deep Merge Support for Array Fields in .cecli.conf.yml

[szmania]

Problem Statement

Currently, both .cecli.conf.yml and .cecli/conf.yml perform shallow merges on array fields. This means when users want to extend configuration arrays (like read, rules, skills_paths, etc.), they must completely override the existing values rather than incrementally adding to them. This creates a poor user experience and makes configuration management difficult.

Project Objectives

• Implement deep merge support for array fields in .cecli.conf.yml
• Maintain backward compatibility with existing .cecli/conf.yml shallow merge behavior
• Support deduplication of array entries
• Preserve order of array elements
• Ensure comprehensive test coverage
• Update documentation

Implementation Details

This PR implements deep merge support for array fields in .cecli.conf.yml files. Key changes include:

• Extended deep_merge function: Modified cecli/helpers/nested.py to handle array concatenation with deduplication and order preservation.
• Array field manifest: Defined DEEP_MERGE_LIST_FIELDS and DEEP_MERGE_JSON_FIELDS to distinguish between top-level list fields and JSON/YAML string fields requiring deep merging.
• Post-parse config merging: Implemented logic in cecli/main.py to manually read and deep-merge .cecli.conf.yml files after configargparse has processed .cecli/conf.yml files. This ensures correct precedence and merge behavior.
• Critical Bugfix: Prevented configargparse from shallow-merging .cecli.conf.yml files by splitting default_config_files into two lists.
• Configuration validation and error handling: Added checks for malformed YAML and incorrect field types, with graceful error logging.
• Comprehensive test coverage: New unit and integration tests cover various merge scenarios, deduplication, order preservation, nested arrays, shallow merge preservation, error handling, performance, and deep copy behavior.

Test Cases Covered

• Deep merge of simple arrays (skills_paths, subagent_paths, tools_paths, allowed_commands)
• Deep merge with duplicates (deduplication preserves first occurrence)
• Order preservation when merging arrays
• Nested array merging (e.g., mcp_servers with nested arrays)
• Shallow merge preservation for .cecli/conf.yml
• Backward compatibility with existing configs
• Error handling for malformed YAML
• Performance with large configurations (< 100ms for 1000+ elements)
• Deep copy behavior (no reference sharing)
• Mixed array and scalar field handling
• Empty array handling
• Null and missing value handling
• Multiple config file stacking (3+ files)
• agent-config JSON string deep merge
• All identified array fields coverage
• Regression tests for existing single config file behaviors
• JSON field serialization after deep merge

This PR addresses CLI-57 and aims to significantly improve the user experience for managing cecli configurations.