rogue/3x-ui-agent
1
0
Fork 0
Code Issues Pull Requests Actions 56 Packages Projects Releases 12 Wiki Activity

ITEM6 B1: sing-box data model + one-shot migration

Browse Source 
New stored model (legacy columns untouched, read paths unchanged
until B2):
- inbounds.singbox_config (converted sing-box inbound/endpoint
  fragment JSON; empty = blocked or external-runtime) and
  inbounds.singbox_preserved (unmapped items verbatim + panel-side
  material + notes)
- agent_nodes.singbox_outbounds / singbox_routing_rules

Converter (internal/singbox, pure functions, ITEM6-CONVERSION-SPEC):
all panel protocols mapped; unmapped constructs are preserved and
registered, never dropped. B0 golden fixtures: 26/26 semantic match
(fragment + unmapped concept/pending sets + disposition).

One-shot migrator (internal/database, InitDB hook after initModels):
marker-gated and idempotent; SQLite file backup before any write
(failure refuses migration/startup); stats switches migrate out of
the template BEFORE the xrayTemplateConfig row is discarded
(SCOPE 4.3.1 order); identity layer is the only client truth with
spec 7-1 mirror reconciliation (adopt/fill/conflict all reported);
fallbacks merged from table + embedded settings, deduped; agent
overrides converted with per-entry issues; everything in one
transaction, marker written last; JSON report written next to the
DB. Malformed rows/templates are hard failures (rollback, old DB
untouched).

settings.trafficStatsSwitches added with factory defaults
(inbound on, outbound off) in defaultValueMap.

Validation: synthetic agent-era DB (mirror drift, fallbacks, xhttp,
agent geo rule, non-default stats) converts as specified; old
production backup (25 inbounds) migrates with credentials, Reality
pbk derivation, traffic counters and reset cycles byte-identical;
idempotent re-run is a no-op; touched packages show zero new test
failures vs the 9-red baseline.
This commit is contained in:
Muse
2026-10-22 19:32:15 +00:00
parent 9843a7ce4e
commit 39ad8b8a2e
12 changed files with 3866 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/database/db.go
+8 -1
View File
@@ -2133,6 +2133,14 @@ func InitDB(dbPath string) error {
return err
}
// ITEM6 B1: one-shot xray→sing-box data migration. No-ops on
// already-migrated databases (settings marker); on failure the
// transaction has rolled back and startup is refused rather than
// running against a half-converted database.
if err := runSingboxMigration(dbPath); err != nil {
return err
}
isUsersEmpty, err := isTableEmpty("users")
if err != nil {
return err
@@ -2279,7 +2287,6 @@ func IsSQLiteDB(file io.ReaderAt) (bool, error) {
return bytes.Equal(buf, signature), nil
}
func ValidateSQLiteDB(dbPath string) error {
if _, err := os.Stat(dbPath); err != nil {
return err
internal/database/model/agent_node.go
+10 -3
View File
@@ -28,9 +28,16 @@ type AgentNode struct {
// RoutingRules is the full xray "routing.rules" JSON array pushed to this
// agent, replacing the template defaults. Empty means "use template defaults".
RoutingRules string `json:"routingRules" gorm:"type:text"`
LastSeen int64 `json:"lastSeen"`
CreatedAt int64 `json:"createdAt" gorm:"autoCreateTime:milli"`
UpdatedAt int64 `json:"updatedAt" gorm:"autoUpdateTime:milli"`
// SingboxOutbounds / SingboxRoutingRules are the sing-box conversions
// of the two override columns above, produced by the B1 one-shot
// migration (ITEM6 §5.6). Entries that cannot be mapped stay only in
// the legacy columns and are listed in the migration report. The
// legacy columns stay the read source until B2.
SingboxOutbounds string `json:"-" gorm:"column:singbox_outbounds;type:text"`
SingboxRoutingRules string `json:"-" gorm:"column:singbox_routing_rules;type:text"`
LastSeen int64 `json:"lastSeen"`
CreatedAt int64 `json:"createdAt" gorm:"autoCreateTime:milli"`
UpdatedAt int64 `json:"updatedAt" gorm:"autoUpdateTime:milli"`
// Heartbeat telemetry (P13): the agent piggybacks host/runtime facts on
// its 60s heartbeat and the panel overlays them onto the node list row.
internal/database/model/model.go
+11
View File
@@ -80,6 +80,17 @@ type Inbound struct {
DisableFlow bool `json:"disableFlow" form:"disableFlow" gorm:"column:disable_flow;default:false" example:"false"`
// SingboxConfig is the sing-box fragment (inbound or endpoint object,
// JSON) converted from the legacy xray-shaped columns by the B1
// one-shot migration (ITEM6). Empty means "no core fragment" — a
// blocked conversion (xhttp/kcp/removed cipher/tunnel) or an
// external-runtime protocol; SingboxPreserved explains which.
// The legacy columns above stay the read source until B2 switches
// generation; these two columns are written only by the migrator
// (and, from B2 on, by the save path). Not exposed via the API yet.
SingboxConfig string `json:"-" gorm:"column:singbox_config;type:text"`
SingboxPreserved string `json:"-" gorm:"column:singbox_preserved;type:text"`
// OriginNodeGuid is the panelGuid of the node that physically hosts this
// inbound, propagated up across hops (#4983). Empty for an inbound that
// lives on this panel's own xray; set to the originating node's GUID when
internal/database/singbox_migrate.go
+727
View File
@@ -0,0 +1,727 @@
package database
// One-shot xray→sing-box data migration (ITEM6 B1).
//
// Runs once from InitDB, after initModels: when the settings marker
// "singboxMigrationVersion" is absent the database still holds the
// legacy xray-shaped model as its only truth. The migration
//
// 1. backs the database file up first (SQLite only; a failed backup
// refuses the migration and therefore startup),
// 2. migrates the template's four traffic-stats switches into their
// own settings key BEFORE the xrayTemplateConfig row is discarded
// (ITEM6-SCOPE §4.3.1/§4.3.2 — the order is load-bearing),
// 3. converts every inbound into a sing-box fragment + preserved
// sidecar (inbounds.singbox_config / singbox_preserved) using the
// identity layer (clients + client_inbounds) as the only truth,
// with the settings.clients mirror reconciled per spec §7-1,
// 4. converts agent-level outbound / routing overrides,
// 5. writes the marker LAST, all inside one transaction — any failure
// rolls everything back and InitDB refuses to start.
//
// Legacy columns are never modified (observation-period rule); the only
// deletions are the xrayTemplateConfig settings row (after its stats
// switches have been migrated out) and nothing else. Re-running against
// an already-migrated database is a no-op via the marker.
import (
"encoding/json"
"errors"
"fmt"
"log"
"os"
"strconv"
"strings"
"time"
"github.com/mhsanaei/3x-ui/v3/internal/database/model"
"github.com/mhsanaei/3x-ui/v3/internal/singbox"
"gorm.io/gorm"
)
const (
singboxMigrationVersion = "1"
singboxMigrationVersionKey = "singboxMigrationVersion"
trafficStatsSwitchesKey = "trafficStatsSwitches"
xrayTemplateSettingKey = "xrayTemplateConfig"
)
// FactoryXrayTemplate is the embedded factory-default xray template
// (web/service config.json). The web/service package publishes it here
// from an init function because internal/database cannot embed a file
// outside its own directory. When empty (database-only test binaries),
// the migration report records the template diff as unavailable and
// still retires the stored row.
var FactoryXrayTemplate string
// trafficStatsSwitches is the settings.trafficStatsSwitches payload
// (ITEM6 §4.3.1): the whole-network traffic-accounting switches that
// used to live in the xray template's policy.system.
type trafficStatsSwitches struct {
InboundUplink bool `json:"inboundUplink"`
InboundDownlink bool `json:"inboundDownlink"`
OutboundUplink bool `json:"outboundUplink"`
OutboundDownlink bool `json:"outboundDownlink"`
}
func defaultTrafficStatsSwitches() trafficStatsSwitches {
// Factory defaults (embedded template policy.system): inbound on,
// outbound off.
return trafficStatsSwitches{InboundUplink: true, InboundDownlink: true}
}
type migrationDiscrepancy struct {
InboundID int `json:"inboundId"`
Tag string `json:"tag"`
Email string `json:"email"`
Kind string `json:"kind"` // mirror-only-client | field-filled-from-mirror | credential-conflict | flow-source | wg-allowed-ips
Field string `json:"field,omitempty"`
Identity string `json:"identity,omitempty"`
Mirror string `json:"mirror,omitempty"`
}
type migrationUnmappedEntry struct {
InboundID int `json:"inboundId"`
Tag string `json:"tag"`
Protocol string `json:"protocol"`
Concept string `json:"concept"`
Pending string `json:"pending"`
Material bool `json:"material"`
}
type migrationNoteEntry struct {
InboundID int `json:"inboundId"`
Tag string `json:"tag"`
Text string `json:"text"`
}
type migrationAgentEntry struct {
NodeID int `json:"nodeId"`
Name string `json:"name"`
OutboundIssues []singbox.OverrideIssue `json:"outboundIssues,omitempty"`
RoutingIssues []singbox.OverrideIssue `json:"routingIssues,omitempty"`
}
type singboxMigrationReport struct {
Version string `json:"version"`
StartedAt string `json:"startedAt"`
Database string `json:"database"` // sqlite | postgres
Backup string `json:"backup,omitempty"`
InboundsTotal int `json:"inboundsTotal"`
InboundsByProtocol map[string]int `json:"inboundsByProtocol"`
Dispositions map[string]int `json:"dispositions"`
FragmentsWritten int `json:"fragmentsWritten"`
EndpointsWritten int `json:"endpointsWritten"`
ClientsProjected int `json:"clientsProjected"`
Discrepancies []migrationDiscrepancy `json:"discrepancies,omitempty"`
Unmapped []migrationUnmappedEntry `json:"unmapped,omitempty"`
Notes []migrationNoteEntry `json:"notes,omitempty"`
AgentNodes []migrationAgentEntry `json:"agentNodes,omitempty"`
StatsSource string `json:"statsSource"` // stored-template | factory-default
Stats trafficStatsSwitches `json:"stats"`
TemplatePresent bool `json:"templatePresent"`
TemplateNonDefault []string `json:"templateNonDefaultSections,omitempty"`
TemplateDiffAvailable bool `json:"templateDiffAvailable"`
ForwardingInbounds int `json:"forwardingInbounds"`
TraditionalNodes int64 `json:"traditionalNodes"`
ReportNotes []string `json:"reportNotes,omitempty"`
}
// runSingboxMigration is the InitDB hook. dbPath is the SQLite file path
// ("" for in-memory / temporary databases, where no file backup exists
// to take and nothing persistent can be lost).
func runSingboxMigration(dbPath string) error {
var marker model.Setting
err := db.Where("key = ?", singboxMigrationVersionKey).First(&marker).Error
if err == nil && marker.Value == singboxMigrationVersion {
return nil // already migrated: zero changes (idempotency)
}
if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
return fmt.Errorf("sing-box migration: read marker: %w", err)
}
rep := &singboxMigrationReport{
Version: singboxMigrationVersion,
StartedAt: time.Now().Format(time.RFC3339),
InboundsByProtocol: map[string]int{},
Dispositions: map[string]int{},
}
if IsPostgres() {
rep.Database = "postgres"
} else {
rep.Database = "sqlite"
}
var inboundCount, agentCount int64
if err := db.Model(&model.Inbound{}).Count(&inboundCount).Error; err != nil {
return fmt.Errorf("sing-box migration: count inbounds: %w", err)
}
if err := db.Model(&model.AgentNode{}).Count(&agentCount).Error; err != nil {
return fmt.Errorf("sing-box migration: count agent nodes: %w", err)
}
templatePresent := settingRowExists(xrayTemplateSettingKey)
hasLegacy := inboundCount > 0 || agentCount > 0 || templatePresent
// Backup gate: nothing to lose on an empty/fresh database; with
// legacy content a failed backup refuses the migration (and hence
// startup), leaving the database untouched.
if hasLegacy {
switch {
case IsPostgres():
rep.ReportNotes = append(rep.ReportNotes,
"PostgreSQL 数据库无文件级备份，迁移依赖单事务回滚保护")
case dbPath == "":
rep.ReportNotes = append(rep.ReportNotes,
"临时/内存 SQLite 数据库，无文件备份")
default:
dst := dbPath + ".pre-singbox-" + time.Now().Format("20060102-150405") + ".bak"
if err := BackupSQLite(dst); err != nil {
return fmt.Errorf("sing-box migration refused: pre-migration backup to %s failed: %w", dst, err)
}
rep.Backup = dst
}
}
if err := db.Transaction(func(tx *gorm.DB) error {
return convertAllForSingbox(tx, rep)
}); err != nil {
return fmt.Errorf("sing-box migration failed and was rolled back (database unchanged): %w", err)
}
log.Printf("sing-box migration v%s complete: %d inbounds (%d fragments, %d endpoints), stats from %s, backup %q",
rep.Version, rep.InboundsTotal, rep.FragmentsWritten, rep.EndpointsWritten, rep.StatsSource, rep.Backup)
if dbPath != "" {
reportPath := dbPath + ".singbox-migration-" + time.Now().Format("20060102-150405") + ".json"
data, merr := json.MarshalIndent(rep, "", " ")
if merr == nil {
if werr := os.WriteFile(reportPath, data, 0o644); werr != nil {
log.Printf("sing-box migration: write report %s: %v (migration itself already committed)", reportPath, werr)
}
}
}
return nil
}
func settingRowExists(key string) bool {
var n int64
if err := db.Model(&model.Setting{}).Where("key = ?", key).Count(&n).Error; err != nil {
return false
}
return n > 0
}
func upsertSetting(tx *gorm.DB, key, value string) error {
var s model.Setting
err := tx.Where("key = ?", key).First(&s).Error
if errors.Is(err, gorm.ErrRecordNotFound) {
return tx.Create(&model.Setting{Key: key, Value: value}).Error
}
if err != nil {
return err
}
if s.Value == value {
return nil
}
s.Value = value
return tx.Save(&s).Error
}
// convertAllForSingbox performs the whole conversion inside tx.
func convertAllForSingbox(tx *gorm.DB, rep *singboxMigrationReport) error {
// -- Step 1 (§4.3.1 order): stats switches OUT of the template first.
stats, source, tplPresent, nonDefault, diffOK, err := migrateTemplateStats(tx)
if err != nil {
return err
}
rep.Stats, rep.StatsSource, rep.TemplatePresent = stats, source, tplPresent
rep.TemplateNonDefault, rep.TemplateDiffAvailable = nonDefault, diffOK
statsJSON, err := json.Marshal(stats)
if err != nil {
return err
}
if err := upsertSetting(tx, trafficStatsSwitchesKey, string(statsJSON)); err != nil {
return fmt.Errorf("write %s: %w", trafficStatsSwitchesKey, err)
}
// -- Step 2 (§4.3.2): only now is the template row discarded.
if tplPresent {
if err := tx.Where("key = ?", xrayTemplateSettingKey).Delete(&model.Setting{}).Error; err != nil {
return fmt.Errorf("discard %s row: %w", xrayTemplateSettingKey, err)
}
}
// -- Step 3: inbounds.
var inbounds []model.Inbound
if err := tx.Order("id ASC").Find(&inbounds).Error; err != nil {
return fmt.Errorf("load inbounds: %w", err)
}
rep.InboundsTotal = len(inbounds)
for i := range inbounds {
ib := &inbounds[i]
rep.InboundsByProtocol[string(ib.Protocol)]++
if s := strings.TrimSpace(ib.Forwarding); s != "" && s != "{}" && s != "null" {
rep.ForwardingInbounds++
}
if err := convertOneInbound(tx, ib, rep); err != nil {
return fmt.Errorf("inbound %d (%s): %w", ib.Id, ib.Tag, err)
}
}
// -- Step 4: agent overrides.
var agents []model.AgentNode
if err := tx.Order("id ASC").Find(&agents).Error; err != nil {
return fmt.Errorf("load agent nodes: %w", err)
}
for i := range agents {
an := &agents[i]
entry := migrationAgentEntry{NodeID: an.Id, Name: an.Name}
changed := false
if strings.TrimSpace(an.Outbounds) != "" {
converted, issues, err := singbox.ConvertAgentOutbounds(an.Outbounds)
if err != nil {
return fmt.Errorf("agent node %d outbounds: %w", an.Id, err)
}
an.SingboxOutbounds = converted
entry.OutboundIssues = issues
changed = true
}
if strings.TrimSpace(an.RoutingRules) != "" {
converted, issues, err := singbox.ConvertAgentRoutingRules(an.RoutingRules)
if err != nil {
return fmt.Errorf("agent node %d routing rules: %w", an.Id, err)
}
an.SingboxRoutingRules = converted
entry.RoutingIssues = issues
changed = true
}
if changed {
if err := tx.Model(&model.AgentNode{}).Where("id = ?", an.Id).Updates(map[string]any{
"singbox_outbounds": an.SingboxOutbounds, "singbox_routing_rules": an.SingboxRoutingRules,
}).Error; err != nil {
return fmt.Errorf("agent node %d: %w", an.Id, err)
}
rep.AgentNodes = append(rep.AgentNodes, entry)
}
}
// Informational only: traditional nodes are untouched by B1 (their
// conversion rides the B6 agent conversion).
if err := tx.Model(&model.Node{}).Count(&rep.TraditionalNodes).Error; err != nil {
rep.ReportNotes = append(rep.ReportNotes, "统计传统 nodes 失败: "+err.Error())
}
// -- Step 5: marker LAST.
if err := upsertSetting(tx, singboxMigrationVersionKey, singboxMigrationVersion); err != nil {
return fmt.Errorf("write migration marker: %w", err)
}
return nil
}
// migrateTemplateStats reads the stored template (if any), extracts the
// four policy.system stats switches, and diffs the stored template
// against the factory default (excluding the four migrated keys) so the
// report can name the non-default sections being discarded.
func migrateTemplateStats(tx *gorm.DB) (stats trafficStatsSwitches, source string, present bool, nonDefault []string, diffOK bool, err error) {
stats = defaultTrafficStatsSwitches()
source = "factory-default"
var row model.Setting
ferr := tx.Where("key = ?", xrayTemplateSettingKey).First(&row).Error
if errors.Is(ferr, gorm.ErrRecordNotFound) {
return stats, source, false, nil, FactoryXrayTemplate != "", nil
}
if ferr != nil {
return stats, source, false, nil, false, ferr
}
present = true
var tpl map[string]any
if jerr := json.Unmarshal([]byte(row.Value), &tpl); jerr != nil {
// A stored template that does not parse is a hard failure: its
// stats switches cannot be extracted, and silently falling back
// to factory defaults could flip whole-network accounting.
return stats, source, true, nil, false,
fmt.Errorf("stored %s is not valid JSON, refusing to guess its stats switches: %w", xrayTemplateSettingKey, jerr)
}
source = "stored-template"
if policy, ok := tpl["policy"].(map[string]any); ok {
if sys, ok := policy["system"].(map[string]any); ok {
if v, ok := sys["statsInboundUplink"].(bool); ok {
stats.InboundUplink = v
}
if v, ok := sys["statsInboundDownlink"].(bool); ok {
stats.InboundDownlink = v
}
if v, ok := sys["statsOutboundUplink"].(bool); ok {
stats.OutboundUplink = v
}
if v, ok := sys["statsOutboundDownlink"].(bool); ok {
stats.OutboundDownlink = v
}
}
}
if FactoryXrayTemplate != "" {
var factory map[string]any
if jerr := json.Unmarshal([]byte(FactoryXrayTemplate), &factory); jerr == nil {
nonDefault = diffTemplateSections(tpl, factory)
diffOK = true
}
}
return stats, source, present, nonDefault, diffOK, nil
}
// diffTemplateSections lists top-level template sections whose content
// differs from the factory default, ignoring the four stats switches
// inside policy.system (they are migrated, not discarded).
func diffTemplateSections(stored, factory map[string]any) []string {
strip := func(m map[string]any) map[string]any {
cp := map[string]any{}
for k, v := range m {
cp[k] = v
}
if policy, ok := cp["policy"].(map[string]any); ok {
pol := map[string]any{}
for k, v := range policy {
pol[k] = v
}
if sys, ok := pol["system"].(map[string]any); ok {
s := map[string]any{}
for k, v := range sys {
switch k {
case "statsInboundUplink", "statsInboundDownlink", "statsOutboundUplink", "statsOutboundDownlink":
default:
s[k] = v
}
}
pol["system"] = s
}
cp["policy"] = pol
}
return cp
}
s, f := strip(stored), strip(factory)
keys := map[string]bool{}
for k := range s {
keys[k] = true
}
for k := range f {
keys[k] = true
}
out := []string{}
for k := range keys {
sb, _ := json.Marshal(s[k])
fb, _ := json.Marshal(f[k])
if string(sb) != string(fb) {
out = append(out, k)
}
}
return out
}
// convertOneInbound converts a single inbound row inside tx.
func convertOneInbound(tx *gorm.DB, ib *model.Inbound, rep *singboxMigrationReport) error {
clients, projected, err := resolveInboundClients(tx, ib, rep)
if err != nil {
return err
}
fallbacks, err := resolveInboundFallbacks(tx, ib)
if err != nil {
return err
}
res, err := singbox.ConvertInbound(singbox.Inbound{
Listen: ib.Listen,
Port: ib.Port,
Protocol: string(ib.Protocol),
DisableFlow: ib.DisableFlow,
Settings: json.RawMessage(ib.Settings),
StreamSettings: json.RawMessage(ib.StreamSettings),
Sniffing: json.RawMessage(ib.Sniffing),
Fallbacks: fallbacks,
Clients: clients,
})
if err != nil {
return err
}
fragJSON, err := res.MarshalFragment()
if err != nil {
return err
}
presJSON, err := res.MarshalPreserved()
if err != nil {
return err
}
if err := tx.Model(&model.Inbound{}).Where("id = ?", ib.Id).Updates(map[string]any{
"singbox_config": fragJSON, "singbox_preserved": presJSON,
}).Error; err != nil {
return err
}
rep.Dispositions[res.Disposition]++
rep.ClientsProjected += projected
if res.Fragment != nil {
if res.Endpoint {
rep.EndpointsWritten++
} else {
rep.FragmentsWritten++
}
}
for _, u := range res.Unmapped {
rep.Unmapped = append(rep.Unmapped, migrationUnmappedEntry{
InboundID: ib.Id, Tag: ib.Tag, Protocol: string(ib.Protocol),
Concept: u.Concept, Pending: u.Pending, Material: u.Material,
})
}
for _, n := range res.Notes {
rep.Notes = append(rep.Notes, migrationNoteEntry{InboundID: ib.Id, Tag: ib.Tag, Text: n})
}
return nil
}
// resolveInboundClients implements spec §7-1: the identity layer
// (clients + client_inbounds, ListForInbound semantics) is the truth;
// the settings.clients mirror only fills gaps — whole-client adoption
// when an identity row is missing entirely, per-field fill when a field
// is empty — and every divergence is recorded.
func resolveInboundClients(tx *gorm.DB, ib *model.Inbound, rep *singboxMigrationReport) ([]singbox.Client, int, error) {
type joinedRow struct {
model.ClientRecord
FlowOverride string
}
var rows []joinedRow
err := tx.Table("clients").
Select("clients.*, client_inbounds.flow_override AS flow_override").
Joins("JOIN client_inbounds ON client_inbounds.client_id = clients.id").
Where("client_inbounds.inbound_id = ?", ib.Id).
Order("clients.id ASC").
Find(&rows).Error
if err != nil {
return nil, 0, fmt.Errorf("identity clients: %w", err)
}
// Mirror (settings.clients) parsed twice: as model.Client for the
// credential fields, and raw for alterId (no identity column).
mirrorByEmail := map[string]*model.Client{}
mirrorAlterID := map[string]int{}
mirrorOrder := []string{}
if strings.TrimSpace(ib.Settings) != "" {
var wrap struct {
Clients []model.Client `json:"clients"`
}
if jerr := json.Unmarshal([]byte(ib.Settings), &wrap); jerr != nil {
return nil, 0, fmt.Errorf("settings.clients mirror is not valid JSON: %w", jerr)
}
for i := range wrap.Clients {
key := strings.ToLower(wrap.Clients[i].Email)
if key == "" {
continue
}
if _, dup := mirrorByEmail[key]; !dup {
mirrorOrder = append(mirrorOrder, key)
}
mirrorByEmail[key] = &wrap.Clients[i]
}
var rawWrap struct {
Clients []map[string]any `json:"clients"`
}
if jerr := json.Unmarshal([]byte(ib.Settings), &rawWrap); jerr == nil {
for _, rc := range rawWrap.Clients {
email, _ := rc["email"].(string)
if email == "" {
continue
}
if v, ok := rc["alterId"].(float64); ok {
mirrorAlterID[strings.ToLower(email)] = int(v)
}
}
}
}
out := []singbox.Client{}
seenIdentity := map[string]bool{}
for i := range rows {
rec := &rows[i]
cl := rec.ToClient()
// Effective flow: the link's flow_override is what generation
// uses (ListForInbound overwrites unconditionally). Fall back
// to the identity record's own flow, then the mirror, when the
// override is empty — all three agree in maintained data.
effectiveFlow := rec.FlowOverride
flowSource := "flow_override"
if effectiveFlow == "" && rec.Flow != "" {
effectiveFlow = rec.Flow
flowSource = "identity"
}
cl.Flow = effectiveFlow
key := strings.ToLower(cl.Email)
seenIdentity[key] = true
mc := mirrorByEmail[key]
if mc != nil {
if flowSource != "flow_override" && mc.Flow != "" && effectiveFlow == "" {
cl.Flow = mc.Flow
flowSource = "mirror"
}
if mc.Flow != "" && effectiveFlow != "" && mc.Flow != effectiveFlow {
rep.Discrepancies = append(rep.Discrepancies, migrationDiscrepancy{
InboundID: ib.Id, Tag: ib.Tag, Email: cl.Email, Kind: "flow-source",
Field: "flow", Identity: effectiveFlow + " (" + flowSource + ")", Mirror: mc.Flow,
})
}
fill := func(field string, dst *string, src string) {
if *dst == "" && src != "" {
*dst = src
rep.Discrepancies = append(rep.Discrepancies, migrationDiscrepancy{
InboundID: ib.Id, Tag: ib.Tag, Email: cl.Email,
Kind: "field-filled-from-mirror", Field: field, Mirror: src,
})
} else if *dst != "" && src != "" && *dst != src {
rep.Discrepancies = append(rep.Discrepancies, migrationDiscrepancy{
InboundID: ib.Id, Tag: ib.Tag, Email: cl.Email,
Kind: "credential-conflict", Field: field, Identity: *dst, Mirror: src,
})
}
}
fill("uuid", &cl.ID, mc.ID)
fill("password", &cl.Password, mc.Password)
fill("auth", &cl.Auth, mc.Auth)
fill("security", &cl.Security, mc.Security)
// Per-inbound WireGuard AllowedIPs live in the mirror.
if len(mc.AllowedIPs) > 0 {
if len(cl.AllowedIPs) > 0 && strings.Join(cl.AllowedIPs, ",") != strings.Join(mc.AllowedIPs, ",") {
rep.Discrepancies = append(rep.Discrepancies, migrationDiscrepancy{
InboundID: ib.Id, Tag: ib.Tag, Email: cl.Email, Kind: "wg-allowed-ips",
Field: "allowedIPs",
Identity: strings.Join(cl.AllowedIPs, ","), Mirror: strings.Join(mc.AllowedIPs, ","),
})
}
cl.AllowedIPs = mc.AllowedIPs
}
if cl.PublicKey == "" && mc.PublicKey != "" {
cl.PublicKey = mc.PublicKey
}
if cl.PreSharedKey == "" && mc.PreSharedKey != "" {
cl.PreSharedKey = mc.PreSharedKey
}
if cl.KeepAlive == 0 && mc.KeepAlive != 0 {
cl.KeepAlive = mc.KeepAlive
}
}
out = append(out, toSingboxClient(cl, mirrorAlterID[key]))
}
// Whole-client adoption for mirror entries with no identity row.
for _, key := range mirrorOrder {
if seenIdentity[key] {
continue
}
mc := mirrorByEmail[key]
rep.Discrepancies = append(rep.Discrepancies, migrationDiscrepancy{
InboundID: ib.Id, Tag: ib.Tag, Email: mc.Email, Kind: "mirror-only-client",
})
out = append(out, toSingboxClient(mc, mirrorAlterID[key]))
}
return out, len(out), nil
}
func toSingboxClient(cl *model.Client, alterID int) singbox.Client {
sc := singbox.Client{
Email: cl.Email,
UUID: cl.ID,
Password: cl.Password,
Auth: cl.Auth,
Flow: cl.Flow,
Security: cl.Security,
AlterID: alterID,
PublicKey: cl.PublicKey,
PreSharedKey: cl.PreSharedKey,
AllowedIPs: cl.AllowedIPs,
KeepAlive: cl.KeepAlive,
Secret: cl.Secret,
AdTag: cl.AdTag,
}
if cl.Reverse != nil {
if b, err := json.Marshal(cl.Reverse); err == nil {
sc.Reverse = b
}
}
return sc
}
// resolveInboundFallbacks merges the inbound_fallbacks table (source of
// truth) with any legacy settings-embedded entries, deduplicated by
// (dest, alpn, path) per spec §7-2. Table entries are shaped like the
// stored form; a table row with an empty dest inherits the child
// inbound's listen:port (wildcard listens become 127.0.0.1), matching
// BuildFallbacksJSON's resolution.
func resolveInboundFallbacks(tx *gorm.DB, ib *model.Inbound) ([]json.RawMessage, error) {
out := []json.RawMessage{}
seen := map[string]bool{}
keyOf := func(dest any, alpn, path string) string {
return fmt.Sprintf("%v|%s|%s", dest, alpn, path)
}
var rows []model.InboundFallback
if err := tx.Where("master_id = ?", ib.Id).Order("sort_order ASC, id ASC").Find(&rows).Error; err != nil {
return nil, fmt.Errorf("fallback rows: %w", err)
}
for _, r := range rows {
destStr := strings.TrimSpace(r.Dest)
var destVal any
switch {
case destStr == "":
var child model.Inbound
if err := tx.First(&child, r.ChildId).Error; err != nil {
return nil, fmt.Errorf("fallback child %d: %w", r.ChildId, err)
}
listen := strings.TrimSpace(child.Listen)
if listen == "" || listen == "0.0.0.0" || listen == "::" || listen == "::0" {
listen = "127.0.0.1"
}
destVal = fmt.Sprintf("%s:%d", listen, child.Port)
default:
if n, aerr := strconv.Atoi(destStr); aerr == nil {
destVal = n
} else {
destVal = destStr
}
}
k := keyOf(destVal, r.Alpn, r.Path)
if seen[k] {
continue
}
seen[k] = true
entry := map[string]any{
"name": r.Name, "alpn": r.Alpn, "path": r.Path, "dest": destVal, "xver": r.Xver,
}
b, err := json.Marshal(entry)
if err != nil {
return nil, err
}
out = append(out, b)
}
// Legacy embedded entries (verbatim), skipping table duplicates.
if strings.TrimSpace(ib.Settings) != "" {
var wrap struct {
Fallbacks []json.RawMessage `json:"fallbacks"`
}
if err := json.Unmarshal([]byte(ib.Settings), &wrap); err == nil {
for _, raw := range wrap.Fallbacks {
var probe struct {
Dest any `json:"dest"`
Alpn string `json:"alpn"`
Path string `json:"path"`
}
_ = json.Unmarshal(raw, &probe)
k := keyOf(probe.Dest, probe.Alpn, probe.Path)
if seen[k] {
continue
}
seen[k] = true
out = append(out, raw)
}
}
}
return out, nil
}
internal/database/singbox_migrate_backupdb_test.go
+576
View File
@@ -0,0 +1,576 @@
package database
import (
"encoding/base64"
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"golang.org/x/crypto/curve25519"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
)
// TestSingboxMigration_OldBackupValidation runs the real migration over
// a copy of a production-era backup database and verifies, field by
// field, that nothing the panel depends on moved: identity credentials,
// Reality key material (stored private key → derived pbk equals the
// stored client-side public key), traffic counters and reset cycles,
// and every legacy column byte-for-byte.
//
// Gated on XUI_B1_OLD_BACKUP pointing at the backup .db file so normal
// test runs skip it.
func TestSingboxMigration_OldBackupValidation(t *testing.T) {
src := os.Getenv("XUI_B1_OLD_BACKUP")
if src == "" {
t.Skip("XUI_B1_OLD_BACKUP not set")
}
dir := t.TempDir()
dst := filepath.Join(dir, "backup.db")
copyFile(t, src, dst)
for _, suffix := range []string{"-wal", "-shm"} {
if _, err := os.Stat(src + suffix); err == nil {
copyFile(t, src+suffix, dst+suffix)
}
}
pre := dumpAllTables(t, dst)
preIdentity := identityByInbound(t, dst)
preMirror := mirrorUUIDsByInbound(t, dst)
preTemplateStats := templateStats(t, dst)
preSettings := settingsMap(t, dst)
if err := InitDB(dst); err != nil {
t.Fatalf("InitDB over backup copy: %v", err)
}
if err := CloseDB(); err != nil {
t.Fatalf("close: %v", err)
}
post := dumpAllTables(t, dst)
postSettings := settingsMap(t, dst)
// 1. Legacy content identical apart from the sanctioned deltas:
// the new singbox_* columns on inbounds, the discarded template
// row, and the two new settings keys. Comparison projects each row
// onto the columns that existed pre-migration (InitDB's own legacy
// migrations may add columns), and the three JSON text columns of
// inbounds are compared semantically (legacy migrations may
// normalize their formatting without changing meaning).
assertTableUnchanged(t, pre, post, "clients")
assertTableUnchanged(t, pre, post, "client_inbounds")
assertTableUnchanged(t, pre, post, "client_traffics")
assertInboundsUnchanged(t, pre, post)
assertSettingsDelta(t, preSettings, postSettings)
// 2. Converted fragments carry exactly the identity layer.
gdb := openRaw(t, dst)
var rows []struct {
Id int
Protocol string
SingboxConfig string
SingboxPreserved string
StreamSettings string
Settings string
}
if err := gdb.Table("inbounds").
Select("id, protocol, singbox_config, singbox_preserved, stream_settings, settings").
Order("id ASC").Scan(&rows).Error; err != nil {
t.Fatalf("scan converted inbounds: %v", err)
}
if len(rows) != 25 {
t.Fatalf("inbounds = %d, want 25", len(rows))
}
blocked := 0
for _, r := range rows {
if r.SingboxConfig == "" {
blocked++
t.Errorf("inbound %d (%s) unexpectedly blocked: preserved=%s", r.Id, r.Protocol, r.SingboxPreserved)
continue
}
var frag map[string]any
if err := json.Unmarshal([]byte(r.SingboxConfig), &frag); err != nil {
t.Fatalf("inbound %d fragment not JSON: %v", r.Id, err)
}
want := preIdentity[r.Id]
switch r.Protocol {
case "vless":
got := map[string]bool{}
for _, u := range fragUsers(frag) {
got[u["uuid"]] = true
if u["uuid"] == "" {
t.Errorf("inbound %d: vless user with empty uuid: %v", r.Id, u)
}
}
// Every identity uuid must be projected; extras are only
// legitimate as spec §7-1 mirror adoptions.
for uuid := range want {
if !got[uuid] {
t.Errorf("inbound %d: identity uuid %s missing from fragment", r.Id, uuid)
}
}
for uuid := range got {
if _, inIdentity := want[uuid]; !inIdentity && !preMirror[r.Id][uuid] {
t.Errorf("inbound %d: fragment uuid %s in neither identity nor mirror", r.Id, uuid)
}
}
checkReality(t, r.Id, r.StreamSettings, frag)
case "shadowsocks":
for _, u := range fragUsers(frag) {
wantPW, ok := want[u["name"]]
if !ok {
t.Errorf("inbound %d: ss user %q not in identity", r.Id, u["name"])
} else if u["password"] != wantPW {
t.Errorf("inbound %d: ss user %q password mismatch", r.Id, u["name"])
}
}
if len(fragUsers(frag)) != len(want) {
t.Errorf("inbound %d: ss users %d != identity %d", r.Id, len(fragUsers(frag)), len(want))
}
case "mixed":
for _, u := range fragUsers(frag) {
if wantPW, ok := want[u["username"]]; ok && u["password"] != wantPW {
t.Errorf("inbound %d: mixed user %q password mismatch", r.Id, u["username"])
}
}
}
}
if blocked != 0 {
t.Fatalf("blocked inbounds = %d, want 0 (production profile is all direct)", blocked)
}
// 3. Stats switches equal the pre-migration template's values.
if v, ok := getSettingRaw(t, gdb, trafficStatsSwitchesKey); !ok {
t.Fatalf("trafficStatsSwitches missing after migration")
} else {
var got trafficStatsSwitches
if err := json.Unmarshal([]byte(v), &got); err != nil {
t.Fatalf("parse stats: %v", err)
}
if preTemplateStats != nil && got != *preTemplateStats {
t.Fatalf("stats = %+v, want template's %+v", got, *preTemplateStats)
}
}
if _, ok := getSettingRaw(t, gdb, xrayTemplateSettingKey); ok {
t.Fatalf("template row should be discarded")
}
if v, ok := getSettingRaw(t, gdb, singboxMigrationVersionKey); !ok || v != "1" {
t.Fatalf("marker = %q, %v", v, ok)
}
// 4. Backup + report artifacts exist.
baks, _ := filepath.Glob(dst + ".pre-singbox-*.bak")
if len(baks) != 1 {
t.Fatalf("backup files = %v, want 1", baks)
}
reports, _ := filepath.Glob(dst + ".singbox-migration-*.json")
if len(reports) != 1 {
t.Fatalf("report files = %v, want 1", reports)
}
repData, _ := os.ReadFile(reports[0])
var rep singboxMigrationReport
if err := json.Unmarshal(repData, &rep); err != nil {
t.Fatalf("parse report: %v", err)
}
t.Logf("report: inbounds=%d fragments=%d endpoints=%d clients=%d dispositions=%v discrepancies=%d unmapped=%d notes=%d statsSource=%s templateNonDefault=%v",
rep.InboundsTotal, rep.FragmentsWritten, rep.EndpointsWritten, rep.ClientsProjected,
rep.Dispositions, len(rep.Discrepancies), len(rep.Unmapped), len(rep.Notes),
rep.StatsSource, rep.TemplateNonDefault)
for _, d := range rep.Discrepancies {
t.Logf("discrepancy: inbound=%d email=%s kind=%s field=%s", d.InboundID, d.Email, d.Kind, d.Field)
}
for _, u := range rep.Unmapped {
t.Logf("unmapped: inbound=%d concept=%s pending=%s material=%v", u.InboundID, u.Concept, u.Pending, u.Material)
}
}
// checkReality verifies the fragment's private_key matches the stored
// streamSettings and that its derived public key equals the stored
// client-side publicKey (the pbk subscribers already use).
func checkReality(t *testing.T, inboundID int, streamSettings string, frag map[string]any) {
t.Helper()
var ss map[string]any
if err := json.Unmarshal([]byte(streamSettings), &ss); err != nil {
t.Fatalf("inbound %d stream: %v", inboundID, err)
}
rs, _ := ss["realitySettings"].(map[string]any)
if rs == nil {
return // not a reality inbound
}
privStored, _ := rs["privateKey"].(string)
tlsBlk, _ := frag["tls"].(map[string]any)
reality, _ := tlsBlk["reality"].(map[string]any)
if reality == nil {
t.Errorf("inbound %d: reality block missing from fragment", inboundID)
return
}
if reality["private_key"] != privStored {
t.Errorf("inbound %d: fragment private_key != stored privateKey", inboundID)
}
settings, _ := rs["settings"].(map[string]any)
storedPub, _ := settings["publicKey"].(string)
if storedPub == "" || privStored == "" {
return
}
privBytes, err := decodeB64(privStored)
if err != nil || len(privBytes) != 32 {
t.Errorf("inbound %d: stored privateKey not a 32-byte base64 key", inboundID)
return
}
pubBytes, err := decodeB64(storedPub)
if err != nil || len(pubBytes) != 32 {
t.Errorf("inbound %d: stored publicKey not a 32-byte base64 key", inboundID)
return
}
var priv, pub [32]byte
copy(priv[:], privBytes)
curve25519.ScalarBaseMult(&pub, &priv)
if string(pub[:]) != string(pubBytes) {
t.Errorf("inbound %d: derived pbk does not match stored publicKey", inboundID)
}
}
// decodeB64 accepts the base64 flavours xray uses for Reality keys
// (raw URL encoding without padding, or padded standard/url).
func decodeB64(s string) ([]byte, error) {
for _, enc := range []*base64.Encoding{
base64.RawURLEncoding, base64.URLEncoding,
base64.StdEncoding, base64.RawStdEncoding,
} {
if b, err := enc.DecodeString(s); err == nil {
return b, nil
}
}
return nil, fmt.Errorf("not base64")
}
func fragUsers(frag map[string]any) []map[string]string {
out := []map[string]string{}
if arr, ok := frag["users"].([]any); ok {
for _, e := range arr {
m, _ := e.(map[string]any)
u := map[string]string{}
for k, v := range m {
if s, ok := v.(string); ok {
u[k] = s
}
}
out = append(out, u)
}
}
return out
}
func keys(m map[string]bool) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
// identityByInbound computes, from the PRE-migration copy, each
// inbound's identity credentials (email → uuid for vless/vmess,
// email → password elsewhere) using ListForInbound's join semantics.
func identityByInbound(t *testing.T, path string) map[int]map[string]string {
t.Helper()
gdb := openRaw(t, path)
type row struct {
InboundId int
Email string
UUID string
Password string
Protocol string
}
var rows []row
err := gdb.Table("clients").
Select("client_inbounds.inbound_id AS inbound_id, clients.email AS email, clients.uuid AS uuid, clients.password AS password, '' AS protocol").
Joins("JOIN client_inbounds ON client_inbounds.client_id = clients.id").
Scan(&rows).Error
if err != nil {
t.Fatalf("identity query: %v", err)
}
protoByInbound := map[int]string{}
var ibs []struct {
Id int
Protocol string
}
if err := gdb.Table("inbounds").Select("id, protocol").Scan(&ibs).Error; err != nil {
t.Fatalf("inbound protocols: %v", err)
}
for _, ib := range ibs {
protoByInbound[ib.Id] = ib.Protocol
}
out := map[int]map[string]string{}
for _, r := range rows {
if out[r.InboundId] == nil {
out[r.InboundId] = map[string]string{}
}
switch protoByInbound[r.InboundId] {
case "vless", "vmess":
out[r.InboundId][r.UUID] = r.Email
default:
out[r.InboundId][r.Email] = r.Password
}
}
return out
}
func templateStats(t *testing.T, path string) *trafficStatsSwitches {
t.Helper()
gdb := openRaw(t, path)
v, ok := getSettingRaw(t, gdb, xrayTemplateSettingKey)
if !ok {
return nil
}
var tpl map[string]any
if err := json.Unmarshal([]byte(v), &tpl); err != nil {
t.Fatalf("pre template parse: %v", err)
}
stats := defaultTrafficStatsSwitches()
if policy, ok := tpl["policy"].(map[string]any); ok {
if sys, ok := policy["system"].(map[string]any); ok {
if b, ok := sys["statsInboundUplink"].(bool); ok {
stats.InboundUplink = b
}
if b, ok := sys["statsInboundDownlink"].(bool); ok {
stats.InboundDownlink = b
}
if b, ok := sys["statsOutboundUplink"].(bool); ok {
stats.OutboundUplink = b
}
if b, ok := sys["statsOutboundDownlink"].(bool); ok {
stats.OutboundDownlink = b
}
}
}
return &stats
}
// dumpAllTables returns every table's rows as stringified key/value
// maps (map-based, so it works against pre-migration schemas lacking
// the new columns).
func dumpAllTables(t *testing.T, path string) map[string][]map[string]string {
t.Helper()
gdb, err := gorm.Open(sqlite.Open(path), &gorm.Config{})
if err != nil {
t.Fatalf("dump open: %v", err)
}
defer func() {
if sqlDB, err := gdb.DB(); err == nil {
_ = sqlDB.Close()
}
}()
var tables []string
if err := gdb.Raw("SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' ORDER BY name").Scan(&tables).Error; err != nil {
t.Fatalf("list tables: %v", err)
}
out := map[string][]map[string]string{}
for _, tbl := range tables {
var rows []map[string]any
if err := gdb.Table(tbl).Find(&rows).Error; err != nil {
t.Fatalf("dump %s: %v", tbl, err)
}
rendered := make([]map[string]string, 0, len(rows))
for _, r := range rows {
m := map[string]string{}
for k, v := range r {
m[k] = fmt.Sprintf("%v", v)
}
rendered = append(rendered, m)
}
out[tbl] = rendered
}
return out
}
// rowsByID indexes rows by their id column.
func rowsByID(rows []map[string]string) map[string]map[string]string {
out := map[string]map[string]string{}
for _, r := range rows {
out[r["id"]] = r
}
return out
}
// renderRow renders the given columns of a row canonically.
func renderRow(r map[string]string, cols []string) string {
var sb strings.Builder
for _, c := range cols {
fmt.Fprintf(&sb, "%s=%s;", c, r[c])
}
return sb.String()
}
// assertTableUnchanged requires the pre-migration rows to survive as
// a multiset with identical values on every column that existed
// pre-migration (post rows may carry additional, newly-added
// columns). Works for composite-key tables too.
func assertTableUnchanged(t *testing.T, pre, post map[string][]map[string]string, table string) {
t.Helper()
preRows, ok := pre[table]
if !ok {
return // table did not exist pre-migration (e.g. agent_nodes)
}
if len(post[table]) != len(preRows) {
t.Fatalf("table %s row count changed: %d -> %d", table, len(preRows), len(post[table]))
}
colSet := map[string]bool{}
for _, r := range preRows {
for k := range r {
colSet[k] = true
}
}
cols := make([]string, 0, len(colSet))
for k := range colSet {
cols = append(cols, k)
}
sort.Strings(cols)
project := func(rows []map[string]string) []string {
out := make([]string, 0, len(rows))
for _, r := range rows {
out = append(out, renderRow(r, cols))
}
sort.Strings(out)
return out
}
preP, postP := project(preRows), project(post[table])
for i := range preP {
if preP[i] != postP[i] {
t.Fatalf("table %s row changed:\npre: %s\npost: %s", table, preP[i], postP[i])
}
}
}
// assertInboundsUnchanged is assertTableUnchanged for inbounds, with
// the three legacy JSON text columns compared semantically (other
// InitDB migrations may reformat them) and the new singbox_* columns
// excluded.
func assertInboundsUnchanged(t *testing.T, pre, post map[string][]map[string]string) {
t.Helper()
preRows := pre["inbounds"]
postByID := rowsByID(post["inbounds"])
if len(postByID) != len(preRows) {
t.Fatalf("inbounds row count changed: %d -> %d", len(preRows), len(postByID))
}
jsonCols := map[string]bool{"settings": true, "stream_settings": true, "sniffing": true}
for _, pr := range preRows {
qr, ok := postByID[pr["id"]]
if !ok {
t.Fatalf("inbound id=%s vanished", pr["id"])
}
for k, v := range pr {
if k == "singbox_config" || k == "singbox_preserved" {
continue
}
if jsonCols[k] {
if !jsonSemanticallyEqual(v, qr[k]) {
t.Fatalf("inbound id=%s column %s changed semantically:\npre: %s\npost: %s", pr["id"], k, v, qr[k])
}
continue
}
if qr[k] != v {
t.Fatalf("inbound id=%s column %s changed: %q -> %q", pr["id"], k, v, qr[k])
}
}
}
}
func jsonSemanticallyEqual(a, b string) bool {
if a == b {
return true
}
var av, bv any
if err := json.Unmarshal([]byte(a), &av); err != nil {
return false
}
if err := json.Unmarshal([]byte(b), &bv); err != nil {
return false
}
ab, _ := json.Marshal(av)
bb, _ := json.Marshal(bv)
return string(ab) == string(bb)
}
func settingsMap(t *testing.T, path string) map[string]string {
t.Helper()
gdb := openRaw(t, path)
var rows []struct {
Key string
Value string
}
if err := gdb.Table("settings").Select("key, value").Scan(&rows).Error; err != nil {
t.Fatalf("settings scan: %v", err)
}
out := map[string]string{}
for _, r := range rows {
out[r.Key] = r.Value
}
return out
}
func mirrorUUIDsByInbound(t *testing.T, path string) map[int]map[string]bool {
t.Helper()
gdb := openRaw(t, path)
var rows []struct {
Id int
Settings string
}
if err := gdb.Table("inbounds").Select("id, settings").Scan(&rows).Error; err != nil {
t.Fatalf("mirror scan: %v", err)
}
out := map[int]map[string]bool{}
for _, r := range rows {
set := map[string]bool{}
var wrap struct {
Clients []struct {
ID string `json:"id"`
} `json:"clients"`
}
if err := json.Unmarshal([]byte(r.Settings), &wrap); err == nil {
for _, c := range wrap.Clients {
if c.ID != "" {
set[c.ID] = true
}
}
}
out[r.Id] = set
}
return out
}
func assertSettingsDelta(t *testing.T, preM, postM map[string]string) {
t.Helper()
for k, v := range preM {
if k == xrayTemplateSettingKey {
continue
}
if postM[k] != v {
t.Fatalf("setting %q changed: %q -> %q", k, v, postM[k])
}
}
for k := range postM {
if _, ok := preM[k]; !ok && k != singboxMigrationVersionKey && k != trafficStatsSwitchesKey {
t.Fatalf("unexpected new setting %q", k)
}
}
if _, ok := postM[xrayTemplateSettingKey]; ok {
t.Fatalf("template row should be gone post-migration")
}
}
func copyFile(t *testing.T, src, dst string) {
t.Helper()
data, err := os.ReadFile(src)
if err != nil {
t.Fatalf("read %s: %v", src, err)
}
if err := os.WriteFile(dst, data, 0o644); err != nil {
t.Fatalf("write %s: %v", dst, err)
}
}
internal/database/singbox_migrate_test.go
+530
View File
@@ -0,0 +1,530 @@
package database
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"github.com/mhsanaei/3x-ui/v3/internal/database/model"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
)
// The migrator runs inside InitDB. These tests seed a legacy-shaped
// database file directly (no InitDB), then drive InitDB over it and
// assert the post-migration state.
func seedLegacyDB(t *testing.T, path string, seed func(gdb *gorm.DB)) {
t.Helper()
gdb, err := gorm.Open(sqlite.Open(path), &gorm.Config{})
if err != nil {
t.Fatalf("open seed db: %v", err)
}
if err := gdb.AutoMigrate(
&model.Inbound{}, &model.ClientRecord{}, &model.ClientInbound{},
&model.Setting{}, &model.AgentNode{}, &model.Node{},
&model.InboundFallback{},
); err != nil {
t.Fatalf("seed automigrate: %v", err)
}
seed(gdb)
sqlDB, err := gdb.DB()
if err != nil {
t.Fatalf("seed sql db: %v", err)
}
if err := sqlDB.Close(); err != nil {
t.Fatalf("close seed db: %v", err)
}
}
func openRaw(t *testing.T, path string) *gorm.DB {
t.Helper()
gdb, err := gorm.Open(sqlite.Open(path), &gorm.Config{})
if err != nil {
t.Fatalf("open raw db: %v", err)
}
t.Cleanup(func() {
if sqlDB, err := gdb.DB(); err == nil {
_ = sqlDB.Close()
}
})
return gdb
}
func getSettingRaw(t *testing.T, gdb *gorm.DB, key string) (string, bool) {
t.Helper()
var s model.Setting
err := gdb.Where("key = ?", key).First(&s).Error
if err == gorm.ErrRecordNotFound {
return "", false
}
if err != nil {
t.Fatalf("get setting %s: %v", key, err)
}
return s.Value, true
}
func factoryTemplate(t *testing.T) string {
t.Helper()
data, err := os.ReadFile(filepath.Join("..", "web", "service", "config.json"))
if err != nil {
t.Fatalf("read factory template: %v", err)
}
return string(data)
}
// modifiedTemplate returns the factory template with the stats switches
// flipped to a non-default combination and one extra non-default
// section (routing.domainStrategy), as a stored-template stand-in.
func modifiedTemplate(t *testing.T) string {
t.Helper()
var tpl map[string]any
if err := json.Unmarshal([]byte(factoryTemplate(t)), &tpl); err != nil {
t.Fatalf("parse factory: %v", err)
}
sys := tpl["policy"].(map[string]any)["system"].(map[string]any)
sys["statsInboundUplink"] = true
sys["statsInboundDownlink"] = false
sys["statsOutboundUplink"] = true
sys["statsOutboundDownlink"] = false
tpl["routing"].(map[string]any)["domainStrategy"] = "UseIPv4"
b, err := json.Marshal(tpl)
if err != nil {
t.Fatalf("marshal template: %v", err)
}
return string(b)
}
const realityStream = `{"network":"tcp","security":"reality","tcpSettings":{"acceptProxyProtocol":false,"header":{"type":"none"}},"realitySettings":{"show":false,"xver":0,"target":"www.microsoft.com:443","serverNames":["www.microsoft.com"],"privateKey":"ISIjJCUmJygpKissLS4vMDEyMzQ1Njc4OTo7PD0+P0A=","minClientVer":"","maxClientVer":"","maxTimediff":0,"shortIds":["77aa10f0"],"mldsa65Seed":"","settings":{"publicKey":"PUBKEYDERIVED0000000000000000000000000000000","fingerprint":"chrome","serverName":"","spiderX":"/"}}}`
func seedAgentEraDB(t *testing.T, path string) {
t.Helper()
seedLegacyDB(t, path, func(gdb *gorm.DB) {
// VLESS + Reality master with a fallback row and an embedded
// duplicate of the same fallback.
vless := &model.Inbound{
UserId: 1, Enable: true, Listen: "203.0.113.10", Port: 20443,
Protocol: model.VLESS, Tag: "n1-in-20443-tcp", Remark: "vless-main",
Settings: `{"clients":[{"email":"alice@example.com","id":"11111111-2222-4333-8444-555555555555","flow":"xtls-rprx-vision","password":"mirror-only-pass"},{"email":"bob@example.com","id":"22222222-3333-4444-8555-666666666666"},{"email":"carol@example.com","id":"33333333-4444-4555-8666-777777777777","password":"carol-pass"}],"decryption":"none","encryption":"none","fallbacks":[{"name":"","alpn":"","path":"","dest":8080,"xver":0}]}`,
StreamSettings: realityStream,
Sniffing: `{"enabled":true,"destOverride":["http","tls"],"metadataOnly":false,"routeOnly":false}`,
Forwarding: `{"enabled":true}`,
}
if err := gdb.Create(vless).Error; err != nil {
t.Fatalf("seed vless: %v", err)
}
// alice: identity row exists but password is empty (mirror fills
// it — alice's mirror entry carries a password on purpose).
// bob: identity only. carol: mirror only (adopted).
alice := &model.ClientRecord{Email: "alice@example.com", UUID: "11111111-2222-4333-8444-555555555555", Flow: "xtls-rprx-vision", Enable: true}
bob := &model.ClientRecord{Email: "bob@example.com", UUID: "22222222-3333-4444-8555-666666666666", Enable: true}
if err := gdb.Create(alice).Error; err != nil {
t.Fatalf("seed alice: %v", err)
}
if err := gdb.Create(bob).Error; err != nil {
t.Fatalf("seed bob: %v", err)
}
for _, c := range []*model.ClientRecord{alice, bob} {
if err := gdb.Create(&model.ClientInbound{ClientId: c.Id, InboundId: vless.Id, FlowOverride: c.Flow}).Error; err != nil {
t.Fatalf("seed link: %v", err)
}
}
child := &model.Inbound{
UserId: 1, Enable: true, Listen: "127.0.0.1", Port: 8080,
Protocol: model.VLESS, Tag: "n1-in-8080-tcp", Remark: "fallback-child",
Settings: `{"clients":[]}`, StreamSettings: `{"network":"tcp","security":"none"}`,
}
if err := gdb.Create(child).Error; err != nil {
t.Fatalf("seed child: %v", err)
}
if err := gdb.Create(&model.InboundFallback{MasterId: vless.Id, ChildId: child.Id, Dest: "8080"}).Error; err != nil {
t.Fatalf("seed fallback: %v", err)
}
ss := &model.Inbound{
UserId: 1, Enable: true, Listen: "", Port: 28443,
Protocol: model.Shadowsocks, Tag: "n1-in-28443-tcpudp", Remark: "ss-main",
Settings: `{"method":"2022-blake3-aes-128-gcm","password":"MDEyMzQ1Njc4OWFiY2RlZg==","network":"tcp,udp","ivCheck":false,"clients":[{"email":"ss@example.com","password":"Y2xpZW50LXBhc3MtMDAwMQ=="}]}`,
StreamSettings: `{"network":"tcp,udp","security":"none"}`,
Sniffing: `{"enabled":false,"destOverride":[],"metadataOnly":false,"routeOnly":false}`,
}
if err := gdb.Create(ss).Error; err != nil {
t.Fatalf("seed ss: %v", err)
}
ssClient := &model.ClientRecord{Email: "ss@example.com", Password: "Y2xpZW50LXBhc3MtMDAwMQ==", Enable: true}
if err := gdb.Create(ssClient).Error; err != nil {
t.Fatalf("seed ss client: %v", err)
}
if err := gdb.Create(&model.ClientInbound{ClientId: ssClient.Id, InboundId: ss.Id}).Error; err != nil {
t.Fatalf("seed ss link: %v", err)
}
xhttp := &model.Inbound{
UserId: 1, Enable: true, Listen: "203.0.113.10", Port: 21443,
Protocol: model.VLESS, Tag: "n1-in-21443-tcp", Remark: "vless-xhttp",
Settings: `{"clients":[{"email":"xh@example.com","id":"44444444-5555-4666-8777-888888888888"}],"decryption":"none"}`,
StreamSettings: `{"network":"xhttp","security":"tls","xhttpSettings":{"path":"/xh"},"tlsSettings":{"serverName":"xh.example.com","certificates":[{"certificateFile":"/etc/x-ui/cert/fx.pem","keyFile":"/etc/x-ui/cert/fx.key"}]}}`,
}
if err := gdb.Create(xhttp).Error; err != nil {
t.Fatalf("seed xhttp: %v", err)
}
xhClient := &model.ClientRecord{Email: "xh@example.com", UUID: "44444444-5555-4666-8777-888888888888", Enable: true}
if err := gdb.Create(xhClient).Error; err != nil {
t.Fatalf("seed xh client: %v", err)
}
if err := gdb.Create(&model.ClientInbound{ClientId: xhClient.Id, InboundId: xhttp.Id}).Error; err != nil {
t.Fatalf("seed xh link: %v", err)
}
agent := &model.AgentNode{
Name: "node-a",
Outbounds: `[{"tag":"direct","protocol":"freedom","settings":{}},` +
`{"tag":"block","protocol":"blackhole","settings":{}},` +
`{"tag":"up","protocol":"socks","settings":{"servers":[{"address":"10.0.0.2","port":1080,"users":[{"user":"u1","pass":"p1"}]}]}}]`,
RoutingRules: `[{"type":"field","outboundTag":"direct","domain":["full:example.com","example.org"]},` +
`{"type":"field","outboundTag":"block","ip":["geoip:private"]}]`,
}
if err := gdb.Create(agent).Error; err != nil {
t.Fatalf("seed agent: %v", err)
}
if err := gdb.Create(&model.Node{Name: "legacy-node-1"}).Error; err != nil {
t.Fatalf("seed node: %v", err)
}
if err := gdb.Create(&model.Setting{Key: "xrayTemplateConfig", Value: modifiedTemplate(t)}).Error; err != nil {
t.Fatalf("seed template: %v", err)
}
})
}
func TestSingboxMigration_AgentEraSample(t *testing.T) {
FactoryXrayTemplate = factoryTemplate(t)
dir := t.TempDir()
path := filepath.Join(dir, "x-ui.db")
seedAgentEraDB(t, path)
if err := InitDB(path); err != nil {
t.Fatalf("InitDB (migration): %v", err)
}
t.Cleanup(func() { _ = CloseDB() })
db := GetDB()
// Marker + stats: stats migrated BEFORE the template row went away.
if v, ok := getSettingRaw(t, db, singboxMigrationVersionKey); !ok || v != "1" {
t.Fatalf("marker = %q, %v; want 1", v, ok)
}
statsRaw, ok := getSettingRaw(t, db, trafficStatsSwitchesKey)
if !ok {
t.Fatalf("trafficStatsSwitches missing")
}
var stats trafficStatsSwitches
if err := json.Unmarshal([]byte(statsRaw), &stats); err != nil {
t.Fatalf("parse stats: %v", err)
}
wantStats := trafficStatsSwitches{InboundUplink: true, InboundDownlink: false, OutboundUplink: true, OutboundDownlink: false}
if stats != wantStats {
t.Fatalf("stats = %+v, want %+v", stats, wantStats)
}
if _, ok := getSettingRaw(t, db, xrayTemplateSettingKey); ok {
t.Fatalf("xrayTemplateConfig row should be discarded after stats migration")
}
// VLESS fragment: reality key carried, 3 users (2 identity + carol
// adopted), fallback registered exactly once despite the duplicate.
var vless model.Inbound
if err := db.Where("port = ?", 20443).First(&vless).Error; err != nil {
t.Fatalf("load vless: %v", err)
}
var frag map[string]any
if err := json.Unmarshal([]byte(vless.SingboxConfig), &frag); err != nil {
t.Fatalf("vless fragment not JSON: %v (%q)", err, vless.SingboxConfig)
}
if frag["type"] != "vless" {
t.Fatalf("fragment type = %v", frag["type"])
}
tlsBlk := frag["tls"].(map[string]any)
reality := tlsBlk["reality"].(map[string]any)
if reality["private_key"] != "ISIjJCUmJygpKissLS4vMDEyMzQ1Njc4OTo7PD0+P0A=" {
t.Fatalf("reality private_key = %v", reality["private_key"])
}
users := frag["users"].([]any)
if len(users) != 3 {
t.Fatalf("users = %d, want 3 (alice, bob, carol adopted)", len(users))
}
var pres struct {
Unmapped []struct {
Concept string `json:"concept"`
} `json:"unmapped"`
Panel map[string]any `json:"panel"`
}
if err := json.Unmarshal([]byte(vless.SingboxPreserved), &pres); err != nil {
t.Fatalf("preserved not JSON: %v", err)
}
fbCount := 0
for _, u := range pres.Unmapped {
if strings.HasPrefix(u.Concept, "fallbacks") {
fbCount++
}
}
if fbCount != 1 {
t.Fatalf("fallback unmapped entries = %d, want 1 (table row + embedded dup deduped)", fbCount)
}
if pres.Panel["realityClientSettings"] == nil {
t.Fatalf("reality client settings should be panel-preserved")
}
// SS fragment.
var ss model.Inbound
if err := db.Where("port = ?", 28443).First(&ss).Error; err != nil {
t.Fatalf("load ss: %v", err)
}
var ssFrag map[string]any
if err := json.Unmarshal([]byte(ss.SingboxConfig), &ssFrag); err != nil {
t.Fatalf("ss fragment: %v", err)
}
if ssFrag["method"] != "2022-blake3-aes-128-gcm" || ssFrag["password"] != "MDEyMzQ1Njc4OWFiY2RlZg==" {
t.Fatalf("ss fragment wrong: %v", ssFrag)
}
if _, hasNetwork := ssFrag["network"]; hasNetwork {
t.Fatalf("ss tcp,udp should be omitted (sing-box default)")
}
// xhttp: blocked, no fragment, preserved explains.
var xh model.Inbound
if err := db.Where("port = ?", 21443).First(&xh).Error; err != nil {
t.Fatalf("load xhttp: %v", err)
}
if xh.SingboxConfig != "" {
t.Fatalf("xhttp fragment should be empty, got %q", xh.SingboxConfig)
}
if !strings.Contains(xh.SingboxPreserved, "xhttp") {
t.Fatalf("xhttp preserved should register the transport: %q", xh.SingboxPreserved)
}
// Agent overrides converted; the geo routing rule is reported, the
// plain rule converted.
var agent model.AgentNode
if err := db.First(&agent).Error; err != nil {
t.Fatalf("load agent: %v", err)
}
if !strings.Contains(agent.SingboxOutbounds, `"type":"direct"`) ||
!strings.Contains(agent.SingboxOutbounds, `"type":"block"`) ||
!strings.Contains(agent.SingboxOutbounds, `"server":"10.0.0.2"`) {
t.Fatalf("agent singbox outbounds wrong: %q", agent.SingboxOutbounds)
}
if strings.Contains(agent.SingboxRoutingRules, "geoip") {
t.Fatalf("geo rule must not be converted: %q", agent.SingboxRoutingRules)
}
if !strings.Contains(agent.SingboxRoutingRules, `"domain":["example.com"]`) {
t.Fatalf("converted routing rule wrong: %q", agent.SingboxRoutingRules)
}
// Backup + report files exist next to the DB.
baks, _ := filepath.Glob(path + ".pre-singbox-*.bak")
if len(baks) != 1 {
t.Fatalf("backup files = %v, want exactly 1", baks)
}
reports, _ := filepath.Glob(path + ".singbox-migration-*.json")
if len(reports) != 1 {
t.Fatalf("report files = %v, want exactly 1", reports)
}
repData, err := os.ReadFile(reports[0])
if err != nil {
t.Fatalf("read report: %v", err)
}
var rep singboxMigrationReport
if err := json.Unmarshal(repData, &rep); err != nil {
t.Fatalf("parse report: %v", err)
}
if rep.InboundsTotal != 4 || rep.StatsSource != "stored-template" {
t.Fatalf("report wrong: total=%d statsSource=%q", rep.InboundsTotal, rep.StatsSource)
}
sawRouting := false
for _, s := range rep.TemplateNonDefault {
if s == "routing" {
sawRouting = true
}
if s == "policy" {
t.Fatalf("policy must not count as non-default once its stats keys are migrated out")
}
}
if !sawRouting {
t.Fatalf("report templateNonDefault = %v, want routing listed", rep.TemplateNonDefault)
}
kinds := map[string]int{}
for _, d := range rep.Discrepancies {
kinds[d.Kind]++
}
if kinds["mirror-only-client"] != 1 {
t.Fatalf("discrepancy kinds = %v, want one mirror-only-client (carol)", kinds)
}
if len(rep.AgentNodes) != 1 || len(rep.AgentNodes[0].RoutingIssues) != 1 {
t.Fatalf("agent report wrong: %+v", rep.AgentNodes)
}
if rep.ForwardingInbounds != 1 || rep.TraditionalNodes != 1 {
t.Fatalf("report counts wrong: forwarding=%d nodes=%d", rep.ForwardingInbounds, rep.TraditionalNodes)
}
}
func snapshotTables(t *testing.T, path string) string {
t.Helper()
gdb := openRaw(t, path)
var sb strings.Builder
var inbounds []model.Inbound
if err := gdb.Order("id ASC").Find(&inbounds).Error; err != nil {
t.Fatalf("snapshot inbounds: %v", err)
}
for _, ib := range inbounds {
b, _ := json.Marshal(ib)
sb.Write(b)
sb.WriteByte('\n')
}
var settings []model.Setting
if err := gdb.Order("key ASC").Find(&settings).Error; err != nil {
t.Fatalf("snapshot settings: %v", err)
}
for _, s := range settings {
sb.WriteString(s.Key + "=" + s.Value + "\n")
}
var agents []model.AgentNode
if err := gdb.Order("id ASC").Find(&agents).Error; err != nil {
t.Fatalf("snapshot agents: %v", err)
}
for _, a := range agents {
b, _ := json.Marshal(a)
sb.Write(b)
sb.WriteByte('\n')
}
return sb.String()
}
func TestSingboxMigration_Idempotent(t *testing.T) {
FactoryXrayTemplate = factoryTemplate(t)
dir := t.TempDir()
path := filepath.Join(dir, "x-ui.db")
seedAgentEraDB(t, path)
if err := InitDB(path); err != nil {
t.Fatalf("first InitDB: %v", err)
}
if err := CloseDB(); err != nil {
t.Fatalf("close: %v", err)
}
first := snapshotTables(t, path)
if err := InitDB(path); err != nil {
t.Fatalf("second InitDB: %v", err)
}
if err := CloseDB(); err != nil {
t.Fatalf("close 2: %v", err)
}
second := snapshotTables(t, path)
if first != second {
t.Fatalf("second run changed the database\nfirst:\n%s\nsecond:\n%s", first, second)
}
baks, _ := filepath.Glob(path + ".pre-singbox-*.bak")
if len(baks) != 1 {
t.Fatalf("second run must not take another backup, got %v", baks)
}
}
func TestSingboxMigration_BadInboundRowRefusesStart(t *testing.T) {
FactoryXrayTemplate = factoryTemplate(t)
dir := t.TempDir()
path := filepath.Join(dir, "x-ui.db")
seedLegacyDB(t, path, func(gdb *gorm.DB) {
bad := &model.Inbound{
UserId: 1, Enable: true, Port: 30443, Protocol: model.VLESS,
Tag: "bad-1", Settings: `{"clients": [broken`,
StreamSettings: `{"network":"tcp","security":"none"}`,
}
if err := gdb.Create(bad).Error; err != nil {
t.Fatalf("seed bad: %v", err)
}
if err := gdb.Create(&model.Setting{Key: "xrayTemplateConfig", Value: modifiedTemplate(t)}).Error; err != nil {
t.Fatalf("seed template: %v", err)
}
})
err := InitDB(path)
if err == nil {
_ = CloseDB()
t.Fatalf("InitDB should refuse to start on a malformed inbound row")
}
_ = CloseDB()
// Old database content is untouched: no marker, template intact, no
// converted fragments.
gdb := openRaw(t, path)
if _, ok := getSettingRaw(t, gdb, singboxMigrationVersionKey); ok {
t.Fatalf("marker must not be written on a failed migration")
}
if _, ok := getSettingRaw(t, gdb, xrayTemplateSettingKey); !ok {
t.Fatalf("template row must survive a failed migration")
}
var ib model.Inbound
if err := gdb.First(&ib).Error; err != nil {
t.Fatalf("load inbound: %v", err)
}
if ib.SingboxConfig != "" || ib.SingboxPreserved != "" {
t.Fatalf("failed migration must leave no converted state, got config=%q preserved=%q", ib.SingboxConfig, ib.SingboxPreserved)
}
// The pre-migration backup was still taken before the attempt.
baks, _ := filepath.Glob(path + ".pre-singbox-*.bak")
if len(baks) != 1 {
t.Fatalf("backup files = %v, want 1", baks)
}
}
func TestSingboxMigration_BadTemplateRefusesStart(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "x-ui.db")
seedLegacyDB(t, path, func(gdb *gorm.DB) {
if err := gdb.Create(&model.Setting{Key: "xrayTemplateConfig", Value: "not-json{"}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
})
if err := InitDB(path); err == nil {
_ = CloseDB()
t.Fatalf("InitDB should refuse to start when the stored template cannot be parsed")
}
_ = CloseDB()
gdb := openRaw(t, path)
if _, ok := getSettingRaw(t, gdb, singboxMigrationVersionKey); ok {
t.Fatalf("marker must not be written on a failed migration")
}
if v, ok := getSettingRaw(t, gdb, xrayTemplateSettingKey); !ok || v != "not-json{" {
t.Fatalf("template row must be untouched, got %q", v)
}
}
func TestSingboxMigration_FreshDatabase(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "x-ui.db")
if err := InitDB(path); err != nil {
t.Fatalf("InitDB fresh: %v", err)
}
t.Cleanup(func() { _ = CloseDB() })
db := GetDB()
if v, ok := getSettingRaw(t, db, singboxMigrationVersionKey); !ok || v != "1" {
t.Fatalf("fresh db marker = %q, %v", v, ok)
}
statsRaw, ok := getSettingRaw(t, db, trafficStatsSwitchesKey)
if !ok {
t.Fatalf("fresh db should record the factory-default stats switches")
}
var stats trafficStatsSwitches
if err := json.Unmarshal([]byte(statsRaw), &stats); err != nil {
t.Fatalf("parse stats: %v", err)
}
if stats != defaultTrafficStatsSwitches() {
t.Fatalf("fresh stats = %+v, want factory defaults", stats)
}
// No legacy content: no backup taken.
baks, _ := filepath.Glob(path + ".pre-singbox-*.bak")
if len(baks) != 0 {
t.Fatalf("fresh db must not produce backups, got %v", baks)
}
}
internal/singbox/convert.go
+905
View File
@@ -0,0 +1,905 @@
// Package singbox converts the panel's legacy xray-shaped inbound model
// into the sing-box fragment shapes defined by ITEM6-CONVERSION-SPEC
// (~/workspace/xui-lite-work/ITEM6-CONVERSION-SPEC.md).
//
// The converter is a pure function over already-resolved inputs: the
// caller (the B1 one-shot migrator in internal/database) resolves the
// identity-layer clients for an inbound, reconciles them against the
// legacy settings.clients mirror, and hands everything in. Anything in
// the xray model that has no sing-box counterpart is NEVER dropped
// silently: it is reported in Result.Unmapped (verbatim input preserved)
// or in Result.Panel (panel-side material such as client-side TLS /
// Reality parameters that subscription generation still needs).
package singbox
import (
"encoding/json"
"fmt"
"net"
"strconv"
"strings"
"time"
)
// Client is one identity-layer client resolved for a specific inbound.
type Client struct {
Email string
UUID string // vless / vmess "id"
Password string // trojan / shadowsocks / mixed / socks / http
Auth string // hysteria auth credential
Flow string // effective flow (flow_override already applied)
Security string // vmess security
AlterID int // vmess alterId (identity layer has no column; mirror-sourced)
// VLESS reverse proxy settings, verbatim JSON when present.
Reverse json.RawMessage
// WireGuard peer material.
PublicKey string
PreSharedKey string
AllowedIPs []string
KeepAlive int
// MTProto (external runtime; preserved only).
Secret string
AdTag string
}
// Inbound is the converter input for one inbound row.
type Inbound struct {
Listen string
Port int
Protocol string
DisableFlow bool // xray DisableInboundFlow switch value for this inbound
Settings json.RawMessage
StreamSettings json.RawMessage
Sniffing json.RawMessage
// Fallbacks holds the resolved fallback entries for vless/vmess/trojan
// masters (inbound_fallbacks table rows merged with any legacy
// settings-embedded entries, deduplicated by the caller).
Fallbacks []json.RawMessage
Clients []Client
}
// Unmapped is one source construct with no sing-box counterpart. Input is
// the verbatim source material so nothing is lost; Pending names the
// decision that owns the item (Q1 user decisions, B2 base-config, ...).
type Unmapped struct {
Concept string `json:"concept"`
Input json.RawMessage `json:"input,omitempty"`
Reason string `json:"reason"`
Pending string `json:"pending"`
// Material reports whether this instance actually loses behaviour
// (a non-default value with nowhere to go), as opposed to a
// default-valued or merely relocated item. Drives Disposition.
Material bool `json:"-"`
}
// Result is the conversion outcome for one inbound.
type Result struct {
// Disposition mirrors the B0 fixture vocabulary: direct,
// partial-pending-q1, blocked-pending-q1, external-runtime.
Disposition string
// Fragment is the sing-box inbound (or endpoint, when Endpoint is
// true) object. Nil when the inbound produces no core fragment
// (blocked conversions and external-runtime protocols).
Fragment map[string]any
Endpoint bool
Unmapped []Unmapped
// Panel holds panel-side preserved material (client-side TLS /
// Reality parameters, externalProxy, passthrough settings, ...).
Panel map[string]any
Notes []string
}
// Preserved is the stored shape of inbounds.singbox_preserved.
type Preserved struct {
Unmapped []Unmapped `json:"unmapped,omitempty"`
Panel map[string]any `json:"panel,omitempty"`
Notes []string `json:"notes,omitempty"`
}
// MarshalPreserved renders the preserved sidecar JSON ("" when empty).
func (r *Result) MarshalPreserved() (string, error) {
if len(r.Unmapped) == 0 && len(r.Panel) == 0 && len(r.Notes) == 0 {
return "", nil
}
p := Preserved{Unmapped: r.Unmapped, Panel: r.Panel, Notes: r.Notes}
b, err := json.Marshal(p)
if err != nil {
return "", err
}
return string(b), nil
}
// MarshalFragment renders the fragment JSON ("" when there is none).
func (r *Result) MarshalFragment() (string, error) {
if r.Fragment == nil {
return "", nil
}
b, err := json.Marshal(r.Fragment)
if err != nil {
return "", err
}
return string(b), nil
}
// ---------------------------------------------------------------------
// small JSON helpers (values are encoding/json generic: map[string]any,
// []any, string, float64, bool, nil)
func parseObj(raw json.RawMessage) (map[string]any, error) {
if len(raw) == 0 {
return map[string]any{}, nil
}
var m map[string]any
if err := json.Unmarshal(raw, &m); err != nil {
return nil, err
}
if m == nil {
m = map[string]any{}
}
return m, nil
}
func rawOf(v any) json.RawMessage {
if v == nil {
return nil
}
if raw, ok := v.(json.RawMessage); ok {
return raw
}
b, err := json.Marshal(v)
if err != nil {
return nil
}
return b
}
func gStr(m map[string]any, key string) string {
if s, ok := m[key].(string); ok {
return s
}
return ""
}
func gBool(m map[string]any, key string) bool {
b, _ := m[key].(bool)
return b
}
func gNum(m map[string]any, key string) (float64, bool) {
switch v := m[key].(type) {
case float64:
return v, true
case json.Number:
f, err := v.Float64()
return f, err == nil
}
return 0, false
}
func gInt(m map[string]any, key string) int {
if f, ok := gNum(m, key); ok {
return int(f)
}
return 0
}
func gArr(m map[string]any, key string) []any {
if a, ok := m[key].([]any); ok {
return a
}
return nil
}
func gObj(m map[string]any, key string) map[string]any {
if o, ok := m[key].(map[string]any); ok {
return o
}
return nil
}
func has(m map[string]any, key string) bool {
_, ok := m[key]
return ok
}
func strArr(v any) []string {
out := []string{}
switch a := v.(type) {
case []any:
for _, e := range a {
if s, ok := e.(string); ok {
out = append(out, s)
}
}
case string:
if a != "" {
out = append(out, a)
}
}
return out
}
// normalizeFlow mirrors the generation-side flow normalisation.
func normalizeFlow(flow string) string {
if flow == "xtls-rprx-vision-udp443" {
return "xtls-rprx-vision"
}
return flow
}
// removedShadowsocksCiphers mirrors model.IsRemovedShadowsocksMethod's
// removed set (stream ciphers dropped by xray and absent from sing-box).
func removedShadowsocksCipher(method string) bool {
switch strings.ToLower(strings.TrimSpace(method)) {
case "aes-128-cfb", "aes-192-cfb", "aes-256-cfb",
"aes-128-ctr", "aes-192-ctr", "aes-256-ctr",
"rc4", "rc4-md5", "bf-cfb", "cast5-cfb", "des-cfb", "idea-cfb",
"seed-cfb", "camellia-128-cfb", "camellia-192-cfb", "camellia-256-cfb",
"salsa20", "chacha20", "chacha20-ietf":
return true
}
return false
}
type conv struct {
in Inbound
set map[string]any
strm map[string]any
res Result
}
func (c *conv) unmapped(concept string, input any, reason, pending string, material bool) {
c.res.Unmapped = append(c.res.Unmapped, Unmapped{
Concept: concept, Input: rawOf(input), Reason: reason,
Pending: pending, Material: material,
})
}
func (c *conv) note(format string, args ...any) {
c.res.Notes = append(c.res.Notes, fmt.Sprintf(format, args...))
}
func (c *conv) panel(key string, v any) {
if c.res.Panel == nil {
c.res.Panel = map[string]any{}
}
c.res.Panel[key] = v
}
// ConvertInbound converts one inbound. A non-nil error means the source
// JSON itself is malformed — the migrator treats that as a hard failure
// (refuse to start) rather than guessing.
func ConvertInbound(in Inbound) (Result, error) {
settings, err := parseObj(in.Settings)
if err != nil {
return Result{}, fmt.Errorf("settings: %w", err)
}
stream, err := parseObj(in.StreamSettings)
if err != nil {
return Result{}, fmt.Errorf("streamSettings: %w", err)
}
c := &conv{in: in, set: settings, strm: stream}
switch in.Protocol {
case "vless":
err = c.convertVLESS()
case "vmess":
err = c.convertVMess()
case "trojan":
err = c.convertTrojan()
case "shadowsocks":
err = c.convertShadowsocks()
case "mixed", "socks":
err = c.convertMixedSocks()
case "http":
err = c.convertHTTP()
case "hysteria":
err = c.convertHysteria()
case "wireguard":
err = c.convertWireguard()
case "tun":
err = c.convertTun()
case "tunnel":
c.convertTunnel()
case "mtproto", "amneziawg":
c.convertExternal()
default:
return Result{}, fmt.Errorf("unsupported protocol %q", in.Protocol)
}
if err != nil {
return Result{}, err
}
c.finishDisposition()
return c.res, nil
}
func (c *conv) finishDisposition() {
switch c.in.Protocol {
case "mtproto", "amneziawg":
c.res.Disposition = "external-runtime"
return
}
if c.res.Fragment == nil {
c.res.Disposition = "blocked-pending-q1"
return
}
for _, u := range c.res.Unmapped {
if u.Material {
c.res.Disposition = "partial-pending-q1"
return
}
}
c.res.Disposition = "direct"
}
// base builds the common fragment head. WireGuard endpoints and tun
// inbounds do not take listen/listen_port and build their own head.
func (c *conv) base(typ string) map[string]any {
frag := map[string]any{
"type": typ,
"tag": fmt.Sprintf("inbound-%d", c.in.Port),
}
if c.in.Listen != "" {
frag["listen"] = c.in.Listen
}
if c.in.Port != 0 {
frag["listen_port"] = c.in.Port
}
// streamSettings.sockopt listen-side options live at inbound level.
c.applySockopt(frag)
return frag
}
// applySockopt maps the mappable sockopt fields onto the fragment and
// registers the rest. ip6Only handling follows the spec's B1 rule: when
// true and the listen is a wildcard, the listen lands on the IPv6
// wildcard; a specific listen is left untouched (and noted).
func (c *conv) applySockopt(frag map[string]any) {
so := gObj(c.strm, "sockopt")
if so == nil {
return
}
rest := map[string]any{}
for k, v := range so {
switch k {
case "mark":
if n, ok := v.(float64); ok && n != 0 {
frag["routing_mark"] = int(n)
}
case "tcpFastOpen":
if b, ok := v.(bool); ok && b {
frag["tcp_fast_open"] = true
}
case "tcpMptcp":
if b, ok := v.(bool); ok && b {
frag["tcp_multi_path"] = true
}
case "interface":
if s, ok := v.(string); ok && s != "" {
frag["bind_interface"] = s
}
case "V6Only", "ip6Only":
if b, ok := v.(bool); ok && b {
if c.in.Listen == "" || c.in.Listen == "0.0.0.0" {
frag["listen"] = "::"
c.note("sockopt.V6Only=true 且监听为通配：listen 落 IPv6 通配 ::")
} else {
c.note("sockopt.V6Only=true 但 listen 已指定为 %s，保持不动", c.in.Listen)
}
}
default:
// Every other sockopt key (tcpcongestion, domainStrategy,
// happyEyeballs, tcpKeepAlive*, V6Only, tproxy, ...) has no
// sing-box listen-side counterpart; it only counts when it
// carries a non-zero value.
if !isZeroValue(v) {
rest[k] = v
}
}
}
if len(rest) > 0 {
c.unmapped("streamSettings.sockopt 无对应字段组", rest,
"sing-box 监听选项无对应字段；原值保留，待 Q1 汇总确认。", "Q1", true)
}
}
func isZeroValue(v any) bool {
switch t := v.(type) {
case nil:
return true
case bool:
return !t
case float64:
return t == 0
case string:
return t == ""
case []any:
return len(t) == 0
case map[string]any:
return len(t) == 0
}
return false
}
// applyStream fills transport / tls / sniff / finalmask / externalProxy
// for the stream-based protocols. Returns false when the transport is a
// blocking form (xhttp / kcp): the fragment must be discarded.
func (c *conv) applyStream(frag map[string]any, proto string) bool {
network := gStr(c.strm, "network")
security := gStr(c.strm, "security")
switch network {
case "xhttp":
c.unmapped("streamSettings.network=xhttp + xhttpSettings", gObj(c.strm, "xhttpSettings"),
"sing-box 无 XHTTP 传输；带该传输的入站整体保留待 Q1（含第三方客户端兼容风险）。", "Q1", true)
return false
case "kcp":
c.unmapped("streamSettings.network=kcp + kcpSettings", gObj(c.strm, "kcpSettings"),
"sing-box 无 mKCP 传输；带该传输的入站整体保留待 Q1。", "Q1", true)
return false
}
if t := c.buildTransport(network); t != nil {
frag["transport"] = t
}
switch security {
case "tls":
if tls := c.buildTLS(gObj(c.strm, "tlsSettings"), false); tls != nil {
frag["tls"] = tls
}
case "reality":
if tls := c.buildReality(gObj(c.strm, "realitySettings")); tls != nil {
frag["tls"] = tls
}
}
c.applySniffing(frag, proto)
c.applyFinalmask(frag, proto)
if ep := gArr(c.strm, "externalProxy"); len(ep) > 0 {
c.panel("externalProxy", ep)
c.note("streamSettings.externalProxy %d 条：订阅/分享侧字段，生成前被剥离，面板侧保留（对应 hosts 表行已由既有迁移生成）", len(ep))
}
return true
}
func (c *conv) buildTransport(network string) map[string]any {
switch network {
case "", "tcp", "raw":
tcp := gObj(c.strm, "tcpSettings")
if tcp == nil {
return nil
}
if gBool(tcp, "acceptProxyProtocol") {
c.unmapped("streamSettings.tcpSettings.acceptProxyProtocol",
map[string]any{"acceptProxyProtocol": true},
"sing-box 监听侧无 PROXY protocol 开关对应，待 Q1。", "Q1", true)
}
hdr := gObj(tcp, "header")
if hdr == nil || gStr(hdr, "type") != "http" {
return nil
}
return c.buildHTTPHeaderTransport(hdr, true)
case "ws":
ws := gObj(c.strm, "wsSettings")
if ws == nil {
return nil
}
if gBool(ws, "acceptProxyProtocol") {
c.unmapped("streamSettings.wsSettings.acceptProxyProtocol",
map[string]any{"acceptProxyProtocol": true},
"sing-box 监听侧无 PROXY protocol 开关对应，待 Q1。", "Q1", true)
}
if hp := gInt(ws, "heartbeatPeriod"); hp > 0 {
c.unmapped("streamSettings.wsSettings.heartbeatPeriod", hp,
"sing-box ws 传输无心跳周期字段，小损登记待 Q1 汇总。", "Q1", true)
}
t := map[string]any{"type": "ws"}
if p := gStr(ws, "path"); p != "" {
t["path"] = p
}
if h := gObj(ws, "headers"); len(h) > 0 {
t["headers"] = h
}
if med := gInt(ws, "maxEarlyData"); med > 0 {
t["max_early_data"] = med
if hn := gStr(ws, "earlyDataHeaderName"); hn != "" {
t["early_data_header_name"] = hn
}
}
return t
case "grpc":
g := gObj(c.strm, "grpcSettings")
if g == nil {
return nil
}
t := map[string]any{"type": "grpc"}
if sn := gStr(g, "serviceName"); sn != "" {
t["service_name"] = sn
}
if gBool(g, "multiMode") {
c.unmapped("streamSettings.grpcSettings.multiMode", true,
"sing-box grpc 传输无 multiMode 对应，小损登记待 Q1 汇总。", "Q1", true)
}
if a := gStr(g, "authority"); a != "" {
c.unmapped("streamSettings.grpcSettings.authority", a,
"sing-box grpc 服务端不承载 authority，小损登记待 Q1 汇总。", "Q1", true)
}
if n := gInt(g, "idle_timeout"); n > 0 {
t["idle_timeout"] = (time.Duration(n) * time.Second).String()
}
if n := gInt(g, "idleTimeout"); n > 0 {
t["idle_timeout"] = (time.Duration(n) * time.Second).String()
}
if n := gInt(g, "pingTimeout"); n > 0 {
t["ping_timeout"] = (time.Duration(n) * time.Second).String()
}
return t
case "httpupgrade":
h := gObj(c.strm, "httpupgradeSettings")
if h == nil {
return nil
}
if gBool(h, "acceptProxyProtocol") {
c.unmapped("streamSettings.httpupgradeSettings.acceptProxyProtocol",
map[string]any{"acceptProxyProtocol": true},
"sing-box 监听侧无 PROXY protocol 开关对应，待 Q1。", "Q1", true)
}
t := map[string]any{"type": "httpupgrade"}
if host := gStr(h, "host"); host != "" {
t["host"] = host
}
if p := gStr(h, "path"); p != "" {
t["path"] = p
}
return t
case "hysteria":
return nil // hysteria transports are protocol-internal
}
return nil
}
// buildHTTPHeaderTransport converts a tcp/ws-style http camouflage header.
// For tcp the request Host is promoted to transport.host; the response
// half has no sing-box server counterpart.
func (c *conv) buildHTTPHeaderTransport(hdr map[string]any, isTCP bool) map[string]any {
req := gObj(hdr, "request")
t := map[string]any{"type": "http"}
if req != nil {
headers := map[string]any{}
if h := gObj(req, "headers"); h != nil {
for k, v := range h {
if strings.EqualFold(k, "host") {
if hosts := strArr(v); len(hosts) > 0 {
t["host"] = hosts
}
continue
}
headers[k] = v
}
}
if p := gArr(req, "path"); len(p) > 0 {
t["path"] = p
} else if ps := gStr(req, "path"); ps != "" {
t["path"] = []string{ps}
}
if m := gStr(req, "method"); m != "" {
t["method"] = m
}
if len(headers) > 0 {
t["headers"] = headers
}
}
if resp := gObj(hdr, "response"); len(resp) > 0 {
c.unmapped("streamSettings.tcpSettings.header.response", resp,
"sing-box 服务端不承载 http 伪装 response 半段，小损登记待 Q1 汇总。", "Q1", true)
}
return t
}
// buildTLS converts tlsSettings into the sing-box tls block. The
// client-side sub-block (settings: fingerprint etc.) is panel material,
// as is disableSystemRoot (client semantics).
func (c *conv) buildTLS(tlsSettings map[string]any, reality bool) map[string]any {
if tlsSettings == nil {
return nil
}
tls := map[string]any{"enabled": true}
if sn := gStr(tlsSettings, "serverName"); sn != "" {
tls["server_name"] = sn
}
if v := gStr(tlsSettings, "minVersion"); v != "" {
tls["min_version"] = v
}
if v := gStr(tlsSettings, "maxVersion"); v != "" {
tls["max_version"] = v
}
if cs := gStr(tlsSettings, "cipherSuites"); cs != "" {
parts := strings.FieldsFunc(cs, func(r rune) bool { return r == ':' || r == ',' })
suites := []string{}
for _, p := range parts {
if p = strings.TrimSpace(p); p != "" {
suites = append(suites, p)
}
}
if len(suites) > 0 {
tls["cipher_suites"] = suites
}
}
if alpn := gArr(tlsSettings, "alpn"); len(alpn) > 0 {
tls["alpn"] = alpn
}
if ech := gArr(tlsSettings, "echConfigList"); len(ech) > 0 {
c.unmapped("tlsSettings.echConfigList（服务端 ECH 配置列表）", ech,
"xray 的 echConfigList 是客户端配置列表，sing-box 服务端 ECH 需私钥派生，查无等价转换，转无对应待 Q1。", "Q1", true)
}
if gBool(tlsSettings, "rejectUnknownSni") {
c.unmapped("tlsSettings.rejectUnknownSni", true,
"sing-box tls 无该开关对应，待 Q1。", "Q1", true)
}
if gBool(tlsSettings, "enableSessionResumption") {
c.unmapped("tlsSettings.enableSessionResumption", true,
"sing-box tls 无会话恢复开关对应，小损登记待 Q1 汇总。", "Q1", true)
}
certs := gArr(tlsSettings, "certificates")
if len(certs) > 0 {
first, _ := certs[0].(map[string]any)
if first != nil {
c.applyCertificate(tls, first)
}
if len(certs) > 1 {
extra := certs[1:]
c.panel("tlsExtraCertificates", extra)
c.note("tlsSettings.certificates 共 %d 对，仅首对直映，其余 %d 对面板侧保留待 Q1", len(certs), len(extra))
}
aux := map[string]any{}
if v := gInt(first0(certs), "ocspStapling"); v != 0 {
aux["ocspStapling"] = v
}
if gBool(first0(certs), "oneTimeLoading") {
aux["oneTimeLoading"] = true
}
if u := gStr(first0(certs), "usage"); u != "" {
aux["usage"] = u
}
if gBool(first0(certs), "buildChain") {
aux["buildChain"] = true
}
if len(aux) > 0 {
c.unmapped("tlsSettings.certificates 附属字段（ocspStapling/oneTimeLoading/usage/buildChain）", aux,
"sing-box 证书加载无这些附属语义，小损登记待 Q1 汇总。", "Q1", true)
}
}
if s := gObj(tlsSettings, "settings"); len(s) > 0 {
c.panel("tlsClientSettings", s)
}
if gBool(tlsSettings, "disableSystemRoot") {
c.panel("tlsDisableSystemRoot", true)
}
return tls
}
func first0(certs []any) map[string]any {
if len(certs) == 0 {
return nil
}
m, _ := certs[0].(map[string]any)
return m
}
func (c *conv) applyCertificate(tls map[string]any, cert map[string]any) {
cf, kf := gStr(cert, "certificateFile"), gStr(cert, "keyFile")
if cf != "" || kf != "" {
if cf != "" {
tls["certificate_path"] = cf
}
if kf != "" {
tls["key_path"] = kf
}
return
}
if certLines := strArr(cert["certificate"]); len(certLines) > 0 {
tls["certificate"] = strings.Join(certLines, "\n")
}
if keyLines := strArr(cert["key"]); len(keyLines) > 0 {
tls["key"] = strings.Join(keyLines, "\n")
}
}
// buildReality converts realitySettings into tls{reality}. The nested
// client-side settings block (publicKey/fingerprint/spiderX) is panel
// material for subscription generation.
func (c *conv) buildReality(rs map[string]any) map[string]any {
if rs == nil {
return nil
}
tls := map[string]any{"enabled": true}
names := strArr(rs["serverNames"])
if len(names) > 0 {
tls["server_name"] = names[0]
if len(names) > 1 {
c.panel("realityExtraServerNames", names[1:])
c.note("realitySettings.serverNames 共 %d 个，sing-box 服务端 server_name 只取首项，其余面板侧保留", len(names))
}
}
target := gStr(rs, "target")
if target == "" {
target = gStr(rs, "dest") // dest is target's historical alias
}
hs := map[string]any{}
if host, port, err := net.SplitHostPort(target); err == nil {
hs["server"] = host
if p, err := strconv.Atoi(port); err == nil {
hs["server_port"] = p
}
} else if target != "" {
hs["server"] = target
}
reality := map[string]any{"enabled": true, "handshake": hs}
if pk := gStr(rs, "privateKey"); pk != "" {
reality["private_key"] = pk
}
if sids := strArr(rs["shortIds"]); len(sids) > 0 {
reality["short_id"] = sids
}
if td, ok := gNum(rs, "maxTimediff"); ok && td > 0 {
// xray counts maxTimediff in milliseconds; sing-box takes a
// duration string. Approximate conversion (spec §4.3, N10 note).
reality["max_time_difference"] = (time.Duration(td) * time.Millisecond).String()
c.note("realitySettings.maxTimediff=%vms 按近似换算为 duration %s", td, reality["max_time_difference"])
}
tls["reality"] = reality
adv := map[string]any{}
if gBool(rs, "show") {
adv["show"] = true
}
if v := gInt(rs, "xver"); v != 0 {
adv["xver"] = v
}
if v := gStr(rs, "minClientVer"); v != "" {
adv["minClientVer"] = v
}
if v := gStr(rs, "maxClientVer"); v != "" {
adv["maxClientVer"] = v
}
if v := gStr(rs, "mldsa65Seed"); v != "" {
adv["mldsa65Seed"] = v
}
if v := gStr(rs, "masterKeyLog"); v != "" {
adv["masterKeyLog"] = v
}
if len(adv) > 0 {
c.unmapped("realitySettings 高级字段（N10）", adv,
"sing-box reality 服务端无这些字段对应（mldsa65Seed 查无等价落点），原值保留待 Q1。", "Q1", true)
}
if s := gObj(rs, "settings"); len(s) > 0 {
c.panel("realityClientSettings", s)
}
return tls
}
// applySniffing mirrors the observed generation-era semantics captured
// by the B0 fixtures: sniff=true lands only for vless/vmess/trojan;
// other protocols accept-and-drop it. destOverride/fakedns and
// metadataOnly have no counterpart and are registered.
func (c *conv) applySniffing(frag map[string]any, proto string) {
var sn map[string]any
if len(c.in.Sniffing) > 0 {
var err error
sn, err = parseObj(c.in.Sniffing)
if err != nil {
// Sniffing is stored as its own JSON text; a malformed blob
// is surfaced as an unmapped note rather than failing the
// whole conversion — generation treats unreadable sniffing
// as disabled too. The verbatim text is preserved.
c.panel("sniffingRaw", string(c.in.Sniffing))
c.note("sniffing JSON 损坏，原文保留在面板侧，转换按未启用处理")
return
}
}
if sn == nil || !gBool(sn, "enabled") {
return
}
switch proto {
case "vless", "vmess", "trojan":
frag["sniff"] = true
}
over := strArr(sn["destOverride"])
for _, d := range over {
if d == "fakedns" {
c.unmapped("sniffing.destOverride 含 fakedns", sn["destOverride"],
"sing-box sniff 无 fakedns 覆盖目标对应，待 Q1。", "Q1", true)
}
}
if gBool(sn, "metadataOnly") {
c.unmapped("sniffing.metadataOnly", true,
"sing-box sniff 无 metadataOnly 对应，待 Q1。", "Q1", true)
}
if gBool(sn, "routeOnly") {
c.note("sniffing.routeOnly=true：sing-box sniff 覆盖即参与路由，近似无损（记录备查）")
}
}
// applyFinalmask handles streamSettings.finalmask. The one direct
// mapping is hysteria2's salamander obfs (consumed by convertHysteria
// before this is called — it sets frag["obfs"] and marks the entry
// consumed); everything else is registered per entry.
func (c *conv) applyFinalmask(frag map[string]any, proto string) {
fm := gObj(c.strm, "finalmask")
if fm == nil {
return
}
for _, section := range []string{"tcp", "udp"} {
for _, e := range gArr(fm, section) {
em, _ := e.(map[string]any)
if em == nil {
continue
}
if proto == "hysteria" && section == "udp" && gStr(em, "type") == "salamander" && frag["obfs"] != nil {
continue // consumed as obfs
}
c.unmapped(fmt.Sprintf("streamSettings.finalmask.%s[%s]", section, gStr(em, "type")), e,
"sing-box 无 finalmask 机制；存量分享链接可能携带 mask 参数，删之属用户可见变化，待 Q1。", "Q1", true)
}
}
if qp := gObj(fm, "quicParams"); len(qp) > 0 {
c.unmapped("streamSettings.finalmask.quicParams", qp,
"sing-box 无 finalmask quicParams 对应，待 Q1。", "Q1", true)
}
}
// salamanderObfs extracts the hysteria2 obfs block from finalmask.udp.
func (c *conv) salamanderObfs() map[string]any {
fm := gObj(c.strm, "finalmask")
if fm == nil {
return nil
}
for _, e := range gArr(fm, "udp") {
em, _ := e.(map[string]any)
if em == nil || gStr(em, "type") != "salamander" {
continue
}
obfs := map[string]any{"type": "salamander"}
if st := gObj(em, "settings"); st != nil {
if pw := gStr(st, "password"); pw != "" {
obfs["password"] = pw
}
}
return obfs
}
return nil
}
// registerFallbacks emits one unmapped entry per resolved fallback.
func (c *conv) registerFallbacks() {
for _, fb := range c.in.Fallbacks {
c.unmapped("fallbacks（inbound_fallbacks 表行 → settings.fallbacks 注入形）", fb,
"sing-box 无 fallback 概念；入站本体直映，回落关系（按 alpn/path 分流到 dest）无承载，候选承载待 Q1 拍板。", "Q1", true)
}
// limitFallback* throttles fallbacks only; without fallbacks they are
// inert, with fallbacks they ride the same decision.
rs := gObj(c.strm, "realitySettings")
lim := map[string]any{}
if v, ok := gNum(rs, "limitFallbackUpload"); ok && v != 0 {
lim["limitFallbackUpload"] = v
}
if v, ok := gNum(rs, "limitFallbackDownload"); ok && v != 0 {
lim["limitFallbackDownload"] = v
}
if len(lim) > 0 {
c.unmapped("realitySettings.limitFallback*（回落限速，N1 附带）", lim,
"随 fallbacks 一并无承载，待 Q1。", "Q1", true)
}
}
// effectiveFlow applies the generation-side flow rules for one client.
func (c *conv) effectiveFlow(cl Client) string {
if c.in.DisableFlow {
return ""
}
return normalizeFlow(cl.Flow)
}
internal/singbox/convert_golden_test.go
+194
View File
@@ -0,0 +1,194 @@
package singbox
import (
"encoding/json"
"os"
"path/filepath"
"reflect"
"sort"
"testing"
)
// TestConvertGolden drives the B1 converter over the B0 golden fixtures
// (ITEM6-CONVERSION-SPEC). Fragments are compared by semantic JSON deep
// equality; unmapped items are compared as (concept, pending) sets —
// the fixture inputs are redacted (kcp seed="*", xmux trimmed), so the
// verbatim input payloads are not byte-comparable by design. Disposition
// must also match the B0 classification.
func TestConvertGolden(t *testing.T) {
root := filepath.Join("testdata", "golden")
dirs, err := os.ReadDir(root)
if err != nil {
t.Fatalf("read golden dir: %v", err)
}
ran := 0
for _, d := range dirs {
if !d.IsDir() {
continue
}
name := d.Name()
t.Run(name, func(t *testing.T) {
in := convertFixture(t, filepath.Join(root, name))
res, err := ConvertInbound(in)
if err != nil {
t.Fatalf("ConvertInbound: %v", err)
}
exp := loadExpected(t, filepath.Join(root, name, "expected-singbox.json"))
// Fragment: expected carries inbounds[] (+ endpoints[] for
// wireguard). Exactly one fragment object at most.
var gotFrag any
if res.Fragment != nil {
gotFrag = res.Fragment
}
var wantFrag any
key := "inbounds"
if res.Endpoint {
key = "endpoints"
}
if arr, ok := exp[key].([]any); ok && len(arr) > 0 {
wantFrag = arr[0]
}
if !reflect.DeepEqual(normalizeJSON(gotFrag), normalizeJSON(wantFrag)) {
t.Errorf("fragment mismatch:\n got: %s\nwant: %s",
marshalIndent(gotFrag), marshalIndent(wantFrag))
}
// Unmapped: (concept, pending) set equality.
gotSet := map[[2]string]int{}
for _, u := range res.Unmapped {
gotSet[[2]string{u.Concept, u.Pending}]++
}
wantSet := map[[2]string]int{}
if b0, ok := exp["_b0"].(map[string]any); ok {
if ums, ok := b0["unmapped"].([]any); ok {
for _, e := range ums {
em, _ := e.(map[string]any)
wantSet[[2]string{strOf(em["concept"]), strOf(em["pending"])}]++
}
}
if disp := strOf(b0["disposition"]); disp != "" && disp != res.Disposition {
t.Errorf("disposition = %q, want %q", res.Disposition, disp)
}
}
if !reflect.DeepEqual(gotSet, wantSet) {
t.Errorf("unmapped set mismatch:\n got: %v\nwant: %v",
sortedPairs(gotSet), sortedPairs(wantSet))
}
})
ran++
}
if ran != 26 {
t.Fatalf("golden cases ran = %d, want 26", ran)
}
}
func strOf(v any) string {
s, _ := v.(string)
return s
}
func sortedPairs(m map[[2]string]int) [][2]string {
out := make([][2]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Slice(out, func(i, j int) bool {
if out[i][0] != out[j][0] {
return out[i][0] < out[j][0]
}
return out[i][1] < out[j][1]
})
return out
}
func normalizeJSON(v any) any {
if v == nil {
return nil
}
b, err := json.Marshal(v)
if err != nil {
return v
}
var out any
if err := json.Unmarshal(b, &out); err != nil {
return v
}
return out
}
func marshalIndent(v any) string {
b, _ := json.MarshalIndent(v, "", " ")
return string(b)
}
func loadExpected(t *testing.T, path string) map[string]any {
t.Helper()
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read expected: %v", err)
}
var m map[string]any
if err := json.Unmarshal(data, &m); err != nil {
t.Fatalf("parse expected: %v", err)
}
return m
}
// convertFixture builds the converter input from a B0 fixture: the
// fixture's settings.clients mirror stands in for the identity layer
// (B0 built the fixtures from a state where mirror == identity), and
// settings.fallbacks supplies the fallback entries.
func convertFixture(t *testing.T, dir string) Inbound {
t.Helper()
data, err := os.ReadFile(filepath.Join(dir, "input.json"))
if err != nil {
t.Fatalf("read input: %v", err)
}
var root map[string]any
if err := json.Unmarshal(data, &root); err != nil {
t.Fatalf("parse input: %v", err)
}
ib, _ := root["inbound"].(map[string]any)
if ib == nil {
t.Fatalf("input has no inbound object")
}
settings := gObj(ib, "settings")
in := Inbound{
Listen: gStr(ib, "listen"),
Port: gInt(ib, "port"),
Protocol: gStr(ib, "protocol"),
Settings: rawOf(ib["settings"]),
StreamSettings: rawOf(ib["streamSettings"]),
Sniffing: rawOf(ib["sniffing"]),
}
for _, e := range gArr(settings, "fallbacks") {
in.Fallbacks = append(in.Fallbacks, rawOf(e))
}
for _, e := range gArr(settings, "clients") {
cm, _ := e.(map[string]any)
if cm == nil {
continue
}
cl := Client{
Email: gStr(cm, "email"),
UUID: gStr(cm, "id"),
Password: gStr(cm, "password"),
Auth: gStr(cm, "auth"),
Flow: gStr(cm, "flow"),
Security: gStr(cm, "security"),
AlterID: gInt(cm, "alterId"),
PublicKey: gStr(cm, "publicKey"),
PreSharedKey: gStr(cm, "preSharedKey"),
AllowedIPs: strArr(cm["allowedIPs"]),
KeepAlive: gInt(cm, "keepAlive"),
Secret: gStr(cm, "secret"),
AdTag: gStr(cm, "adTag"),
}
if rev, ok := cm["reverse"]; ok && rev != nil {
cl.Reverse = rawOf(rev)
}
in.Clients = append(in.Clients, cl)
}
return in
}
internal/singbox/convert_outbounds.go
+260
View File
@@ -0,0 +1,260 @@
package singbox
import (
"encoding/json"
"fmt"
"strings"
)
// OverrideIssue records one agent-level outbound / routing entry that
// could not be mapped (spec §5.6). The entry stays untouched in the old
// column; nothing is silently dropped.
type OverrideIssue struct {
Index int `json:"index"`
Kind string `json:"kind"` // outbound protocol or "routing-rule"
Detail string `json:"detail"`
}
// ConvertAgentOutbounds maps a stored xray-shaped outbounds array (the
// agent_nodes.outbounds override) into sing-box outbounds. Entries that
// cannot be mapped are omitted from the output and reported as issues.
// Returns ("", nil, nil) for empty input.
func ConvertAgentOutbounds(raw string) (string, []OverrideIssue, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return "", nil, nil
}
var arr []map[string]any
if err := json.Unmarshal([]byte(raw), &arr); err != nil {
return "", nil, fmt.Errorf("outbounds: %w", err)
}
out := []any{}
var issues []OverrideIssue
for i, ob := range arr {
converted, issue := convertOneOutbound(ob)
if issue != "" {
issues = append(issues, OverrideIssue{Index: i, Kind: gStr(ob, "protocol"), Detail: issue})
continue
}
out = append(out, converted)
}
b, err := json.Marshal(out)
if err != nil {
return "", nil, err
}
return string(b), issues, nil
}
func convertOneOutbound(ob map[string]any) (map[string]any, string) {
proto := gStr(ob, "protocol")
tag := gStr(ob, "tag")
settings := gObj(ob, "settings")
switch proto {
case "freedom":
out := map[string]any{"type": "direct"}
if tag != "" {
out["tag"] = tag
}
return out, ""
case "blackhole":
out := map[string]any{"type": "block"}
if tag != "" {
out["tag"] = tag
}
return out, ""
case "socks", "http":
srv := firstServer(settings)
if srv == nil {
return nil, "no servers[0] to map"
}
out := map[string]any{"type": proto}
if tag != "" {
out["tag"] = tag
}
out["server"] = gStr(srv, "address")
if p := gInt(srv, "port"); p != 0 {
out["server_port"] = p
}
if users := gArr(srv, "users"); len(users) > 0 {
if u, _ := users[0].(map[string]any); u != nil {
if name := gStr(u, "user"); name != "" {
out["username"] = name
}
if pass := gStr(u, "pass"); pass != "" {
out["password"] = pass
}
}
}
return out, ""
case "shadowsocks":
srv := firstServer(settings)
if srv == nil {
return nil, "no servers[0] to map"
}
out := map[string]any{"type": "shadowsocks"}
if tag != "" {
out["tag"] = tag
}
out["server"] = gStr(srv, "address")
if p := gInt(srv, "port"); p != 0 {
out["server_port"] = p
}
method := gStr(srv, "method")
if method == "" {
method = gStr(settings, "method")
}
password := gStr(srv, "password")
if password == "" {
password = gStr(settings, "password")
}
if removedShadowsocksCipher(method) {
return nil, "shadowsocks method " + method + " removed in sing-box"
}
out["method"] = method
out["password"] = password
return out, ""
case "vless":
vnext := gArr(settings, "vnext")
if len(vnext) == 0 {
return nil, "no vnext[0] to map"
}
vn, _ := vnext[0].(map[string]any)
if vn == nil {
return nil, "vnext[0] malformed"
}
out := map[string]any{"type": "vless"}
if tag != "" {
out["tag"] = tag
}
out["server"] = gStr(vn, "address")
if p := gInt(vn, "port"); p != 0 {
out["server_port"] = p
}
if users := gArr(vn, "users"); len(users) > 0 {
if u, _ := users[0].(map[string]any); u != nil {
out["uuid"] = gStr(u, "id")
if f := gStr(u, "flow"); f != "" {
out["flow"] = normalizeFlow(f)
}
}
}
return out, ""
default:
return nil, "protocol " + proto + " has no outbound mapping in this batch"
}
}
func firstServer(settings map[string]any) map[string]any {
if settings == nil {
return nil
}
servers := gArr(settings, "servers")
if len(servers) == 0 {
return nil
}
srv, _ := servers[0].(map[string]any)
return srv
}
// ConvertAgentRoutingRules maps stored xray routing rules into sing-box
// route rules. Rules referencing geoip:/geosite: (fully excised in this
// product line) or using unmapped constructs are omitted and reported.
func ConvertAgentRoutingRules(raw string) (string, []OverrideIssue, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return "", nil, nil
}
var arr []map[string]any
if err := json.Unmarshal([]byte(raw), &arr); err != nil {
return "", nil, fmt.Errorf("routing rules: %w", err)
}
out := []any{}
var issues []OverrideIssue
for i, rule := range arr {
converted, issue := convertOneRoutingRule(rule)
if issue != "" {
issues = append(issues, OverrideIssue{Index: i, Kind: "routing-rule", Detail: issue})
continue
}
if converted != nil {
out = append(out, converted)
}
}
b, err := json.Marshal(out)
if err != nil {
return "", nil, err
}
return string(b), issues, nil
}
func convertOneRoutingRule(rule map[string]any) (map[string]any, string) {
if t := gStr(rule, "type"); t != "" && t != "field" {
return nil, "rule type " + t + " not mappable"
}
outbound := gStr(rule, "outboundTag")
if outbound == "" {
return nil, "rule has no outboundTag (balancer/other target)"
}
// Any geo reference anywhere in the rule defeats mapping.
raw, _ := json.Marshal(rule)
if strings.Contains(string(raw), "geoip:") || strings.Contains(string(raw), "geosite:") {
return nil, "rule references geoip:/geosite: (geo excised); kept in old column pending Q1"
}
out := map[string]any{"outbound": outbound}
if tags := strArr(rule["inboundTag"]); len(tags) > 0 {
out["inbound"] = tags
}
if domains := strArr(rule["domain"]); len(domains) > 0 {
var full, suffix, keyword, regex []string
for _, d := range domains {
switch {
case strings.HasPrefix(d, "full:"):
full = append(full, strings.TrimPrefix(d, "full:"))
case strings.HasPrefix(d, "keyword:"):
keyword = append(keyword, strings.TrimPrefix(d, "keyword:"))
case strings.HasPrefix(d, "regexp:"):
regex = append(regex, strings.TrimPrefix(d, "regexp:"))
default:
suffix = append(suffix, d)
}
}
if len(full) > 0 {
out["domain"] = full
}
if len(suffix) > 0 {
out["domain_suffix"] = suffix
}
if len(keyword) > 0 {
out["domain_keyword"] = keyword
}
if len(regex) > 0 {
out["domain_regex"] = regex
}
}
if ips := strArr(rule["ip"]); len(ips) > 0 {
out["ip_cidr"] = ips
}
switch p := rule["port"].(type) {
case float64:
out["port"] = int(p)
case string:
if p != "" {
out["port_range"] = p
}
}
if protos := strArr(rule["protocol"]); len(protos) > 0 {
out["protocol"] = protos
}
if n := gStr(rule, "network"); n != "" {
nets := []string{}
for _, part := range strings.Split(n, ",") {
if part = strings.TrimSpace(part); part != "" {
nets = append(nets, part)
}
}
if len(nets) > 0 {
out["network"] = nets
}
}
return out, ""
}
internal/singbox/convert_protocols.go
+455
View File
@@ -0,0 +1,455 @@
package singbox
import (
"encoding/json"
"fmt"
"net"
"strconv"
)
// ---------------------------------------------------------------- VLESS
func (c *conv) convertVLESS() error {
frag := c.base("vless")
users := make([]any, 0, len(c.in.Clients))
for _, cl := range c.in.Clients {
u := map[string]any{"name": cl.Email, "uuid": cl.UUID}
if f := c.effectiveFlow(cl); f != "" {
u["flow"] = f
}
if len(cl.Reverse) > 0 {
c.unmapped("vless client reverse", map[string]any{
"email": cl.Email, "reverse": jsonRawString(cl.Reverse),
}, "sing-box vless 无 reverse 反向代理对应（N4），整客户其余字段照常直映。", "Q1", true)
}
users = append(users, u)
}
if len(users) > 0 {
frag["users"] = users
}
if dec := gStr(c.set, "decryption"); dec != "" && dec != "none" {
c.unmapped("settings.decryption（vlessenc / ML-KEM 加密串）", dec,
"sing-box vless 无对应加密层；带 vlessenc 的入站迁过去即明文 vless，行为与安全性变化，待 Q1。", "Q1", true)
}
if enc := gStr(c.set, "encryption"); enc != "" && enc != "none" {
c.panel("vlessEncryption", enc)
}
if !c.applyStream(frag, "vless") {
c.res.Fragment = nil
return nil
}
c.registerFallbacks()
c.res.Fragment = frag
return nil
}
// ---------------------------------------------------------------- VMess
func (c *conv) convertVMess() error {
frag := c.base("vmess")
users := make([]any, 0, len(c.in.Clients))
for _, cl := range c.in.Clients {
u := map[string]any{"name": cl.Email, "uuid": cl.UUID}
if cl.Security != "" {
u["security"] = cl.Security
}
u["alter_id"] = cl.AlterID
users = append(users, u)
}
if len(users) > 0 {
frag["users"] = users
}
if !c.applyStream(frag, "vmess") {
c.res.Fragment = nil
return nil
}
c.res.Fragment = frag
return nil
}
// --------------------------------------------------------------- Trojan
func (c *conv) convertTrojan() error {
frag := c.base("trojan")
users := make([]any, 0, len(c.in.Clients))
for _, cl := range c.in.Clients {
u := map[string]any{"name": cl.Email, "password": cl.Password}
if f := c.effectiveFlow(cl); f != "" {
// sing-box trojan users have no flow field; generation writes
// it into the xray user today, so this is a real (if small)
// behavioural item owned by Q1/B3.
c.unmapped("trojan client flow", f,
"sing-box trojan 用户无 flow 字段；flow 客户端（vision）实际协商行为待 Q1/B3 定。", "Q1/B3", true)
}
users = append(users, u)
}
if len(users) > 0 {
frag["users"] = users
}
if !c.applyStream(frag, "trojan") {
c.res.Fragment = nil
return nil
}
c.registerFallbacks()
c.res.Fragment = frag
return nil
}
// ---------------------------------------------------------- Shadowsocks
func (c *conv) convertShadowsocks() error {
method := gStr(c.set, "method")
if removedShadowsocksCipher(method) {
c.unmapped("shadowsocks method="+method, map[string]any{"method": method},
"sing-box 已移除 cfb/ctr 类 cipher，无承载；入站整体保留待 Q1。", "Q1", true)
c.res.Fragment = nil
return nil
}
frag := c.base("shadowsocks")
frag["method"] = method
if pw := gStr(c.set, "password"); pw != "" {
frag["password"] = pw
}
users := make([]any, 0, len(c.in.Clients))
for _, cl := range c.in.Clients {
users = append(users, map[string]any{"name": cl.Email, "password": cl.Password})
}
if len(users) > 0 {
frag["users"] = users
}
// "tcp,udp" is sing-box's default coverage and is omitted; a
// restricted network value is carried over.
switch n := gStr(c.set, "network"); n {
case "", "tcp,udp":
case "tcp", "udp":
frag["network"] = n
default:
c.note("shadowsocks settings.network=%q 非标准值，未落 network 字段，原文在旧列保留", n)
}
if has(c.set, "ivCheck") {
c.note("shadowsocks settings.ivCheck=%v：sing-box 无对应校验开关，登记备查（Q19 口径，待 Q1 汇总确认）", gBool(c.set, "ivCheck"))
}
if !c.applyStream(frag, "shadowsocks") {
c.res.Fragment = nil
return nil
}
c.res.Fragment = frag
return nil
}
// ------------------------------------------------------- mixed / socks
func (c *conv) socksUsers() []any {
users := []any{}
seen := map[string]bool{}
for _, a := range gArr(c.set, "accounts") {
am, _ := a.(map[string]any)
if am == nil {
continue
}
user := gStr(am, "user")
if user == "" {
continue
}
seen[user] = true
users = append(users, map[string]any{"username": user, "password": gStr(am, "pass")})
}
for _, cl := range c.in.Clients {
if cl.Email == "" || seen[cl.Email] {
continue
}
seen[cl.Email] = true
users = append(users, map[string]any{"username": cl.Email, "password": cl.Password})
}
return users
}
func (c *conv) convertMixedSocks() error {
typ := "mixed"
if c.in.Protocol == "socks" {
typ = "socks"
}
frag := c.base(typ)
if gStr(c.set, "auth") != "noauth" {
if users := c.socksUsers(); len(users) > 0 {
frag["users"] = users
}
}
if ip := gStr(c.set, "ip"); ip != "" && ip != "127.0.0.1" {
c.unmapped("settings.ip（UDP 回源地址）", ip,
"sing-box mixed/socks 无回源地址对应；默认值 127.0.0.1 无损，非默认登记待 Q1。", "Q1", true)
}
if has(c.set, "udp") && !gBool(c.set, "udp") {
c.note("settings.udp=false：sing-box mixed/socks 的 UDP 关联恒开，行为差异登记备查")
}
if !c.applyStream(frag, c.in.Protocol) {
c.res.Fragment = nil
return nil
}
c.res.Fragment = frag
return nil
}
// ----------------------------------------------------------------- HTTP
func (c *conv) convertHTTP() error {
frag := c.base("http")
users := []any{}
for _, a := range gArr(c.set, "accounts") {
am, _ := a.(map[string]any)
if am == nil {
continue
}
if user := gStr(am, "user"); user != "" {
users = append(users, map[string]any{"username": user, "password": gStr(am, "pass")})
}
}
if len(users) > 0 {
frag["users"] = users
}
if gBool(c.set, "allowTransparent") {
c.unmapped("settings.allowTransparent", true,
"sing-box http 入站无透明代理开关对应，待 Q1。", "Q1", true)
}
if !c.applyStream(frag, "http") {
c.res.Fragment = nil
return nil
}
c.res.Fragment = frag
return nil
}
// ------------------------------------------------------------- Hysteria
func (c *conv) convertHysteria() error {
version := gInt(c.set, "version")
if hs := gObj(c.strm, "hysteriaSettings"); hs != nil {
if sv := gInt(hs, "version"); sv != 0 && version != 0 && sv != version {
c.note("hysteria 版本真源冲突：settings.version=%d、streamSettings.hysteriaSettings.version=%d，按 settings 为准", version, sv)
}
}
if version == 0 {
version = 2
}
typ := "hysteria2"
if version == 1 {
typ = "hysteria"
}
frag := c.base(typ)
users := make([]any, 0, len(c.in.Clients))
for _, cl := range c.in.Clients {
if version == 1 {
users = append(users, map[string]any{"name": cl.Email, "auth": cl.Auth})
} else {
users = append(users, map[string]any{"name": cl.Email, "password": cl.Auth})
}
}
if len(users) > 0 {
frag["users"] = users
}
if version == 2 {
if obfs := c.salamanderObfs(); obfs != nil {
frag["obfs"] = obfs
}
if mz := gObj(c.set, "masquerade"); mz != nil {
if m := convertMasquerade(mz); m != nil {
frag["masquerade"] = m
}
}
}
if has(c.set, "udpIdleTimeout") && gInt(c.set, "udpIdleTimeout") != 0 {
c.note("hysteria settings.udpIdleTimeout=%d：sing-box 对应字段语义未对齐，小损备查（待 Q1 汇总）", gInt(c.set, "udpIdleTimeout"))
}
if !c.applyStream(frag, "hysteria") {
c.res.Fragment = nil
return nil
}
c.res.Fragment = frag
return nil
}
func convertMasquerade(mz map[string]any) map[string]any {
switch gStr(mz, "type") {
case "file":
out := map[string]any{"type": "file"}
if d := gStr(mz, "dir"); d != "" {
out["directory"] = d
}
return out
case "proxy":
out := map[string]any{"type": "proxy"}
if u := gStr(mz, "url"); u != "" {
out["url"] = u
}
if gBool(mz, "rewriteHost") {
out["rewrite_host"] = true
}
return out
case "string":
out := map[string]any{"type": "string"}
if sc := gInt(mz, "statusCode"); sc != 0 {
out["status_code"] = sc
}
if content := gStr(mz, "content"); content != "" {
out["content"] = content
}
return out
}
return nil
}
// ------------------------------------------------------------ WireGuard
func (c *conv) convertWireguard() error {
frag := map[string]any{
"type": "wireguard",
"tag": fmt.Sprintf("inbound-%d", c.in.Port),
}
c.res.Endpoint = true
if c.in.Port != 0 {
frag["listen_port"] = c.in.Port
}
if addr := deriveWireguardAddress(c.set); addr != "" {
frag["address"] = []string{addr}
} else {
c.note("wireguard settings 缺 subnetIp/subnetCidr，服务端 address 未推导，原文在旧列保留")
}
if sk := gStr(c.set, "secretKey"); sk != "" {
frag["private_key"] = sk
}
if mtu := gInt(c.set, "mtu"); mtu > 0 {
frag["mtu"] = mtu
}
peers := make([]any, 0, len(c.in.Clients))
for _, cl := range c.in.Clients {
p := map[string]any{"public_key": cl.PublicKey}
if len(cl.AllowedIPs) > 0 {
ips := make([]any, len(cl.AllowedIPs))
for i, s := range cl.AllowedIPs {
ips[i] = s
}
p["allowed_ips"] = ips
}
if cl.PreSharedKey != "" {
p["pre_shared_key"] = cl.PreSharedKey
}
if cl.KeepAlive > 0 {
p["persistent_keepalive_interval"] = cl.KeepAlive
}
peers = append(peers, p)
}
if len(peers) > 0 {
frag["peers"] = peers
}
if has(c.set, "noKernelTun") {
c.note("wireguard settings.noKernelTun=%v：sing-box wireguard endpoint 的系统/内核模式差异待 B3 真机验（N7）", gBool(c.set, "noKernelTun"))
}
if !c.applyStream(frag, "wireguard") {
c.res.Fragment = nil
return nil
}
c.res.Fragment = frag
return nil
}
// deriveWireguardAddress mirrors the panel's server-address derivation:
// the first host address of the configured subnet (10.0.0.0/24 ->
// 10.0.0.1/24).
func deriveWireguardAddress(settings map[string]any) string {
subnet := gStr(settings, "subnetIp")
cidr := gInt(settings, "subnetCidr")
if subnet == "" || cidr == 0 {
return ""
}
ip := net.ParseIP(subnet)
if ip == nil {
return ""
}
ip4 := ip.To4()
if ip4 == nil {
return ""
}
mask := net.CIDRMask(cidr, 32)
if mask == nil {
return ""
}
network := ip4.Mask(mask)
host := make(net.IP, 4)
copy(host, network)
host[3]++
return host.String() + "/" + strconv.Itoa(cidr)
}
// ------------------------------------------------------------------ TUN
func (c *conv) convertTun() error {
frag := map[string]any{
"type": "tun",
"tag": fmt.Sprintf("inbound-%d", c.in.Port),
}
if name := gStr(c.set, "name"); name != "" {
frag["interface_name"] = name
}
if gw := c.set["gateway"]; gw != nil {
switch v := gw.(type) {
case []any:
if len(v) > 0 {
frag["address"] = v
}
case string:
if v != "" {
frag["address"] = []string{v}
}
}
}
if mtu := gInt(c.set, "mtu"); mtu > 0 {
frag["mtu"] = mtu
}
if routes := gArr(c.set, "autoSystemRoutingTable"); len(routes) > 0 {
frag["auto_route"] = true
frag["route_address"] = routes
}
frag["stack"] = "system"
if dns := gArr(c.set, "dns"); len(dns) > 0 {
c.unmapped("settings.dns", dns,
"tun 入站本体无 dns 字段，由 sing-box 独立 dns 配置承载——属基础配置面，B2 定落点，此处登记防丢。", "B2", false)
}
if has(c.set, "userLevel") {
lvl := gInt(c.set, "userLevel")
c.unmapped("settings.userLevel", lvl,
"xray policy level 概念，sing-box 无对应；0 为默认无损，非 0 待 Q1。", "Q1", lvl != 0)
}
if aoi := gStr(c.set, "autoOutboundsInterface"); aoi != "" && aoi != "auto" {
c.note("tun settings.autoOutboundsInterface=%q 非 auto，sing-box tun 无直映字段，备查", aoi)
}
// TUN takes no stream/sniff handling in the current generation chain.
c.res.Fragment = frag
return nil
}
// --------------------------------------------------------------- Tunnel
func (c *conv) convertTunnel() {
c.unmapped("tunnel(dokodemo-door) settings", c.set,
"tun/tunnel 在 sing-box 侧的落点（tun 入站 + 转发模块）跨 B2/B3，本批整体保留待 Q1。", "Q1", true)
c.res.Fragment = nil
}
// ------------------------------------------- external-runtime protocols
func (c *conv) convertExternal() {
// MTProto and AmneziaWG run on external runtimes in the current
// architecture; the panel keeps their full settings verbatim.
c.panel("passthrough", c.set)
c.note("协议 %s 由外部运行时承载（现行架构），settings 原样保留在面板侧，本批不产核心片段", c.in.Protocol)
c.res.Fragment = nil
}
// jsonRawString unwraps a raw JSON value for embedding in another object.
func jsonRawString(raw []byte) any {
var v any
if err := json.Unmarshal(raw, &v); err != nil {
return string(raw)
}
return v
}
internal/singbox/convert_test.go
+169
View File
@@ -0,0 +1,169 @@
package singbox
import (
"encoding/json"
"strings"
"testing"
)
func mustConvert(t *testing.T, in Inbound) Result {
t.Helper()
res, err := ConvertInbound(in)
if err != nil {
t.Fatalf("ConvertInbound: %v", err)
}
return res
}
func TestConvert_MaxTimediffBecomesDuration(t *testing.T) {
stream := `{"network":"tcp","security":"reality","realitySettings":{"target":"example.com:443","serverNames":["example.com"],"privateKey":"PK","shortIds":["ab"],"maxTimediff":300000,"settings":{}}}`
res := mustConvert(t, Inbound{
Port: 443, Protocol: "vless",
Settings: json.RawMessage(`{"clients":[]}`),
StreamSettings: json.RawMessage(stream),
Clients: []Client{{Email: "a@b.c", UUID: "u-1"}},
})
tlsBlk := res.Fragment["tls"].(map[string]any)
reality := tlsBlk["reality"].(map[string]any)
if reality["max_time_difference"] != "5m0s" {
t.Fatalf("max_time_difference = %v, want 5m0s", reality["max_time_difference"])
}
}
func TestConvert_CipherSuitesSplitAndInlineCerts(t *testing.T) {
stream := `{"network":"tcp","security":"tls","tlsSettings":{"serverName":"s.example.com","cipherSuites":"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256:TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256","certificates":[{"certificate":["-----BEGIN CERTIFICATE-----","MIIB","-----END CERTIFICATE-----"],"key":["-----BEGIN KEY-----","MIIB","-----END KEY-----"]}]}}`
res := mustConvert(t, Inbound{
Port: 443, Protocol: "trojan",
Settings: json.RawMessage(`{"clients":[]}`),
StreamSettings: json.RawMessage(stream),
Clients: []Client{{Email: "a@b.c", Password: "pw"}},
})
tlsBlk := res.Fragment["tls"].(map[string]any)
suites := tlsBlk["cipher_suites"].([]string)
if len(suites) != 2 || !strings.HasPrefix(suites[0], "TLS_ECDHE_ECDSA") {
t.Fatalf("cipher_suites = %v", suites)
}
if !strings.Contains(tlsBlk["certificate"].(string), "\n") {
t.Fatalf("inline certificate should be joined with newlines: %q", tlsBlk["certificate"])
}
if _, ok := tlsBlk["certificate_path"]; ok {
t.Fatalf("inline certs must not produce certificate_path")
}
}
func TestConvert_DeriveWireguardAddress(t *testing.T) {
cases := []struct {
subnet string
cidr int
want string
}{
{"10.0.0.0", 24, "10.0.0.1/24"},
{"10.9.8.0", 24, "10.9.8.1/24"},
{"172.16.0.0", 16, "172.16.0.1/16"},
{"", 24, ""},
{"not-an-ip", 24, ""},
}
for _, tc := range cases {
m := map[string]any{"subnetIp": tc.subnet, "subnetCidr": float64(tc.cidr)}
if got := deriveWireguardAddress(m); got != tc.want {
t.Errorf("derive(%q,%d) = %q, want %q", tc.subnet, tc.cidr, got, tc.want)
}
}
}
func TestConvert_V6OnlyWildcardListen(t *testing.T) {
stream := `{"network":"tcp","security":"none","sockopt":{"V6Only":true}}`
res := mustConvert(t, Inbound{
Port: 443, Protocol: "vless", Listen: "",
Settings: json.RawMessage(`{"clients":[]}`),
StreamSettings: json.RawMessage(stream),
})
if res.Fragment["listen"] != "::" {
t.Fatalf("listen = %v, want ::", res.Fragment["listen"])
}
}
func TestConvert_SockoptMappingsAndRemainder(t *testing.T) {
stream := `{"network":"tcp","security":"none","sockopt":{"mark":42,"tcpFastOpen":true,"tcpcongestion":"bbr"}}`
res := mustConvert(t, Inbound{
Port: 443, Protocol: "vless",
Settings: json.RawMessage(`{"clients":[]}`),
StreamSettings: json.RawMessage(stream),
})
if res.Fragment["routing_mark"] != 42 || res.Fragment["tcp_fast_open"] != true {
t.Fatalf("sockopt mappings wrong: %v", res.Fragment)
}
found := false
for _, u := range res.Unmapped {
if strings.Contains(u.Concept, "sockopt") && strings.Contains(string(u.Input), "bbr") {
found = true
}
}
if !found {
t.Fatalf("tcpcongestion should be registered unmapped, got %+v", res.Unmapped)
}
}
func TestConvert_DisableFlowClearsUserFlow(t *testing.T) {
res := mustConvert(t, Inbound{
Port: 443, Protocol: "vless", DisableFlow: true,
Settings: json.RawMessage(`{"clients":[]}`),
StreamSettings: json.RawMessage(`{"network":"tcp","security":"none"}`),
Clients: []Client{{Email: "a@b.c", UUID: "u", Flow: "xtls-rprx-vision"}},
})
users := res.Fragment["users"].([]any)
if _, ok := users[0].(map[string]any)["flow"]; ok {
t.Fatalf("flow must be cleared when DisableFlow is set: %v", users[0])
}
}
func TestConvert_UDP443FlowNormalized(t *testing.T) {
res := mustConvert(t, Inbound{
Port: 443, Protocol: "vless",
Settings: json.RawMessage(`{"clients":[]}`),
StreamSettings: json.RawMessage(`{"network":"tcp","security":"none"}`),
Clients: []Client{{Email: "a@b.c", UUID: "u", Flow: "xtls-rprx-vision-udp443"}},
})
users := res.Fragment["users"].([]any)
if users[0].(map[string]any)["flow"] != "xtls-rprx-vision" {
t.Fatalf("flow = %v", users[0])
}
}
func TestConvert_MalformedSettingsIsHardError(t *testing.T) {
_, err := ConvertInbound(Inbound{Port: 1, Protocol: "vless", Settings: json.RawMessage(`{"clients":`)})
if err == nil {
t.Fatalf("malformed settings must be a hard error")
}
}
func TestConvertAgentOutbounds(t *testing.T) {
raw := `[{"tag":"direct","protocol":"freedom","settings":{}},{"tag":"warp","protocol":"wireguard","settings":{}}]`
out, issues, err := ConvertAgentOutbounds(raw)
if err != nil {
t.Fatalf("convert: %v", err)
}
if !strings.Contains(out, `"type":"direct"`) || strings.Contains(out, "warp") {
t.Fatalf("outbounds = %s", out)
}
if len(issues) != 1 || issues[0].Kind != "wireguard" {
t.Fatalf("issues = %+v", issues)
}
if out2, issues2, _ := ConvertAgentOutbounds(""); out2 != "" || issues2 != nil {
t.Fatalf("empty input should produce empty output, got %q %v", out2, issues2)
}
}
func TestConvertAgentRoutingRules_GeoRejected(t *testing.T) {
raw := `[{"type":"field","outboundTag":"block","ip":["geoip:cn"]},{"type":"field","outboundTag":"direct","port":53,"network":"udp"}]`
out, issues, err := ConvertAgentRoutingRules(raw)
if err != nil {
t.Fatalf("convert: %v", err)
}
if len(issues) != 1 {
t.Fatalf("issues = %+v", issues)
}
if !strings.Contains(out, `"port":53`) || strings.Contains(out, "geoip") {
t.Fatalf("rules = %s", out)
}
}
internal/web/service/setting.go
+21 -8
View File
@@ -32,6 +32,13 @@ import (
//go:embed config.json
var xrayTemplateConfig string
// Publish the factory template to the database package: the B1
// migration (internal/database) diffs a stored template against it but
// cannot embed a file outside its own directory.
func init() {
database.FactoryXrayTemplate = xrayTemplateConfig
}
const (
DefaultSubClashUserAgentRegex = `(?i)(clash|mihomo)`
DefaultSubJsonUserAgentRegex = ``
@@ -42,14 +49,20 @@ const (
var defaultValueMap = map[string]string{
"xrayTemplateConfig": xrayTemplateConfig,
"webListen": "",
"webDomain": "",
"webPort": "2053",
"webCertFile": "",
"webKeyFile": "",
"secret": random.Seq(32),
"panelGuid": uuid.NewString(),
"apiToken": "",
// trafficStatsSwitches (ITEM6 §4.3.1): the whole-network traffic
// accounting switches, moved out of the xray template's
// policy.system by the B1 migration; the B1 migrator writes the
// migrated values, this entry is the factory default (inbound on,
// outbound off) for databases that never stored a template.
"trafficStatsSwitches": `{"inboundUplink":true,"inboundDownlink":true,"outboundUplink":false,"outboundDownlink":false}`,
"webListen": "",
"webDomain": "",
"webPort": "2053",
"webCertFile": "",
"webKeyFile": "",
"secret": random.Seq(32),
"panelGuid": uuid.NewString(),
"apiToken": "",
// Node mTLS material (opt-in). All default empty: the CA + master client
// cert are minted lazily on first use, and the node-side trust CA is pasted
// in by the operator. Kept out of entity.AllSetting so private keys never