e2c4ca6d56
Two live gate attempts failed with "An error occurred downloading the FUT
Squad Update" while the redirect response was verified byte-identical to the
Python oracle. Rolling back to the Python redirector fixed it, so the response
bytes were never the whole contract.
Log archaeology found the discriminator: the client polls
/fifa17/fut/rosterupdate.xml ~4x/min in every successful FUT session, and the
only gap in 300 recorded fetches is 03:47-03:58 -- exactly the two
Rust-redirector sessions. The Blaze RPC sequence over those sessions is
identical (msgNum 0-53), so the divergence is entirely outside Blaze.
A differential lifecycle probe against both redirectors found the two
behaviours this host never reproduced:
* the oracle drains the request body per Content-Length; this host stopped
at the header terminator, leaving unread data in the receive queue, which
makes Linux close with RST rather than FIN
* the oracle holds the connection open ~300ms before closing
(time.sleep(0.3)); this host closed at 0ms
Both are now reproduced. The dwell is a named constant, ORACLE_CLOSE_DWELL,
overridable only so the causal experiment -- set it to 0, confirm the failure
returns -- can be run without a rebuild.
The suite could not have caught either: it sent Content-Length: 0, so there
was never a body to drain. It now POSTs a body, and asserts a split-write body
is fully consumed.
Testing the drain via client-visible symptoms does NOT work -- verified by
mutation: with the drain removed the client still reads the buffered response
and sees close_notify before any reset. So the host records a per-connection
ConnOutcome and the test asserts on that. Both mutations (no-dwell, no-drain)
are now each caught by exactly one test.
This does not yet prove causation for the FUT Squad Update failure; it removes
the only two measured divergences. Gate 6 is the test.
376 lines
13 KiB
Rust
376 lines
13 KiB
Rust
//! # openfut-redirector-host
|
|
//!
|
|
//! Transport host for the FIFA 17 Blaze redirector — the first hop.
|
|
//!
|
|
//! ```text
|
|
//! FIFA 17 ──TLS 1.2, static-RSA──> this host ──> <serverinstanceinfo>
|
|
//! "connect to <adv>:<blaze_port>"
|
|
//! ```
|
|
//!
|
|
//! ## A deliberately small compatibility island
|
|
//!
|
|
//! This is the only OpenFUT crate that links OpenSSL, and it does so because a
|
|
//! retail FIFA 17 client offers exactly eight static-RSA suites and nothing
|
|
//! forward-secret. Neither OpenFUT Core nor the generic protocol crates gain
|
|
//! that dependency. See [`tls`] for what the observed ClientHello dictates.
|
|
//!
|
|
//! ## Division of responsibility
|
|
//!
|
|
//! The host owns the listener, TLS, HTTP framing, connection lifecycle and
|
|
//! diagnostics. `openfut-adapter-fifa17::redirector` owns the response and
|
|
//! nothing else — the same split as the Blaze sidecar, where the adapter
|
|
//! decides what to say and the host owns the socket.
|
|
//!
|
|
//! No FUT state lives here.
|
|
//!
|
|
//! ## Configuration
|
|
//!
|
|
//! Resolved through `openfut-host-config`, the single environment reader, so
|
|
//! the advertised address reaches this host by the same construction path as
|
|
//! the Blaze sidecar's.
|
|
|
|
pub mod config;
|
|
pub mod tls;
|
|
|
|
use std::io::{Read, Write};
|
|
use std::net::{TcpListener, TcpStream};
|
|
use std::sync::atomic::{AtomicU64, Ordering};
|
|
use std::sync::Arc;
|
|
use std::time::{Duration, SystemTime, UNIX_EPOCH};
|
|
|
|
use openfut_adapter_fifa17::redirector;
|
|
use openssl::ssl::SslAcceptor;
|
|
|
|
pub use config::RedirectorConfig;
|
|
|
|
fn log(msg: &str) {
|
|
let ms = SystemTime::now()
|
|
.duration_since(UNIX_EPOCH)
|
|
.map(|d| d.as_millis())
|
|
.unwrap_or(0);
|
|
eprintln!("[{}.{:03}] {msg}", ms / 1000, ms % 1000);
|
|
}
|
|
|
|
/// The commit this binary was built from.
|
|
///
|
|
/// Only the commit: a compiled-in cleanliness claim can go stale (cargo will
|
|
/// not re-run a build script for another crate's edit), so the authoritative
|
|
/// comparison happens at launch in `scripts/verify-build-identity.sh`.
|
|
pub const BUILD_COMMIT: &str = env!("OPENFUT_BUILD_COMMIT");
|
|
|
|
/// Build identity, printable without any configuration.
|
|
///
|
|
/// `--identity` exists so the launcher can establish which commit a binary came
|
|
/// from before deciding whether to run it. Machine-readable `key=value`.
|
|
pub fn identity() -> String {
|
|
format!(
|
|
"openfut-redirector-host v{} commit={} profile={} openssl={}",
|
|
env!("CARGO_PKG_VERSION"),
|
|
BUILD_COMMIT,
|
|
if cfg!(debug_assertions) {
|
|
"debug"
|
|
} else {
|
|
"release"
|
|
},
|
|
tls::openssl_version(),
|
|
)
|
|
}
|
|
|
|
/// One line naming the binary, its linked TLS, and what it will advertise.
|
|
///
|
|
/// Machine-readable `key=value` so the launcher can extract the commit without
|
|
/// guessing at prose.
|
|
pub fn banner(cfg: &RedirectorConfig) -> String {
|
|
format!(
|
|
"openfut-redirector-host v{} commit={} profile={} openssl={} listen={} \
|
|
advertise={}:{} tls_min={:?} tls_max={:?} security_level={} ciphers={}",
|
|
env!("CARGO_PKG_VERSION"),
|
|
BUILD_COMMIT,
|
|
if cfg!(debug_assertions) {
|
|
"debug"
|
|
} else {
|
|
"release"
|
|
},
|
|
tls::openssl_version(),
|
|
cfg.listen_on(),
|
|
cfg.adapter.endpoints.advertise,
|
|
cfg.adapter.endpoints.blaze_port,
|
|
cfg.tls.min_version,
|
|
cfg.tls.max_version,
|
|
cfg.tls
|
|
.security_level
|
|
.map(|l| l.to_string())
|
|
.unwrap_or_else(|| "default".into()),
|
|
cfg.tls.cipher_list,
|
|
)
|
|
}
|
|
|
|
/// A bound listener, so a caller can learn the real port before serving
|
|
/// (ephemeral ports in tests) and so the self-test runs before anything is
|
|
/// accepted.
|
|
pub struct Server {
|
|
pub local_addr: std::net::SocketAddr,
|
|
listener: TcpListener,
|
|
acceptor: Arc<SslAcceptor>,
|
|
cfg: Arc<RedirectorConfig>,
|
|
outcomes: Outcomes,
|
|
}
|
|
|
|
/// What the host did on a connection, beyond the bytes it sent.
|
|
///
|
|
/// Recorded because the failure that cost two live gate attempts — a request
|
|
/// body left unread — is invisible to any comparison of the response. The log
|
|
/// line carries the same facts for a live run; this makes them assertable.
|
|
#[derive(Debug, Clone)]
|
|
pub struct ConnOutcome {
|
|
pub id: u64,
|
|
pub request_bytes: usize,
|
|
pub body: BodyRead,
|
|
pub response_bytes: usize,
|
|
}
|
|
|
|
pub type Outcomes = Arc<std::sync::Mutex<Vec<ConnOutcome>>>;
|
|
|
|
/// Keep the record bounded: a long-running redirector must not accumulate one
|
|
/// entry per connection forever.
|
|
const OUTCOME_HISTORY: usize = 64;
|
|
|
|
impl Server {
|
|
/// A handle to the connection record, obtainable before [`Server::run`]
|
|
/// consumes the server.
|
|
pub fn outcomes(&self) -> Outcomes {
|
|
self.outcomes.clone()
|
|
}
|
|
|
|
pub fn run(self) -> std::io::Result<()> {
|
|
let counter = AtomicU64::new(0);
|
|
for incoming in self.listener.incoming() {
|
|
let Ok(stream) = incoming else { continue };
|
|
let id = counter.fetch_add(1, Ordering::Relaxed) + 1;
|
|
let (acceptor, cfg) = (self.acceptor.clone(), self.cfg.clone());
|
|
let outcomes = self.outcomes.clone();
|
|
std::thread::spawn(move || handle(stream, id, &acceptor, &cfg, &outcomes));
|
|
}
|
|
Ok(())
|
|
}
|
|
}
|
|
|
|
fn record(outcomes: &Outcomes, o: ConnOutcome) {
|
|
if let Ok(mut v) = outcomes.lock() {
|
|
if v.len() >= OUTCOME_HISTORY {
|
|
v.remove(0);
|
|
}
|
|
v.push(o);
|
|
}
|
|
}
|
|
|
|
/// Build TLS, rehearse the retail handshake, and bind.
|
|
pub fn bind(cfg: RedirectorConfig) -> std::io::Result<Server> {
|
|
let acceptor = tls::build_acceptor(&cfg.tls)
|
|
.map_err(|e| std::io::Error::other(format!("TLS setup failed: {e}")))?;
|
|
// Rehearse the retail handshake BEFORE accepting anything: a client
|
|
// restricted to exactly the suites FIFA offers must connect. Catching a
|
|
// cipher/version misconfiguration here means it never shows up as an
|
|
// unexplained failure during a live gate.
|
|
let neg = tls::self_test(&cfg.tls, tls::OBSERVED_CLIENT_SUITES).map_err(|e| {
|
|
std::io::Error::other(format!(
|
|
"self-test failed — a FIFA-like client cannot connect: {e}"
|
|
))
|
|
})?;
|
|
log(&format!(
|
|
"SELF-TEST OK: a FIFA-like client negotiates {} / {}",
|
|
neg.version, neg.cipher
|
|
));
|
|
|
|
let listener = TcpListener::bind(cfg.listen_on())?;
|
|
let local_addr = listener.local_addr()?;
|
|
log(&banner(&cfg));
|
|
Ok(Server {
|
|
local_addr,
|
|
listener,
|
|
acceptor: Arc::new(acceptor),
|
|
cfg: Arc::new(cfg),
|
|
outcomes: Outcomes::default(),
|
|
})
|
|
}
|
|
|
|
/// Serve until the process is killed.
|
|
pub fn serve(cfg: RedirectorConfig) -> std::io::Result<()> {
|
|
bind(cfg)?.run()
|
|
}
|
|
|
|
fn handle(
|
|
stream: TcpStream,
|
|
id: u64,
|
|
acceptor: &SslAcceptor,
|
|
cfg: &RedirectorConfig,
|
|
outcomes: &Outcomes,
|
|
) {
|
|
let peer = stream
|
|
.peer_addr()
|
|
.map(|a| a.to_string())
|
|
.unwrap_or_else(|_| "<unknown>".into());
|
|
let _ = stream.set_read_timeout(Some(Duration::from_secs(15)));
|
|
let _ = stream.set_write_timeout(Some(Duration::from_secs(15)));
|
|
|
|
let mut tls = match acceptor.accept(stream) {
|
|
Ok(s) => s,
|
|
Err(e) => {
|
|
// The most valuable diagnostic this host produces: a handshake
|
|
// failure names the client and the reason, so a cipher/version
|
|
// mismatch is obvious rather than looking like a network fault.
|
|
log(&format!("conn-{id:04} {peer} TLS HANDSHAKE FAILED: {e}"));
|
|
return;
|
|
}
|
|
};
|
|
|
|
// Exactly what the retail client negotiated. Recorded per connection
|
|
// because it is the evidence a gate is judged on.
|
|
{
|
|
let s = tls.ssl();
|
|
log(&format!(
|
|
"conn-{id:04} {peer} TLS OK version={} cipher={} sni={}",
|
|
s.version_str(),
|
|
s.current_cipher().map(|c| c.name()).unwrap_or("?"),
|
|
s.servername(openssl::ssl::NameType::HOST_NAME)
|
|
.unwrap_or("<none>")
|
|
));
|
|
}
|
|
|
|
// Read the request head. The oracle answers any request with the same body,
|
|
// so this is parsed for diagnostics, not for routing — a redirector that
|
|
// started 404ing unexpected paths would be a behaviour change, not a fix.
|
|
let mut buf = Vec::new();
|
|
let mut chunk = [0u8; 4096];
|
|
loop {
|
|
match tls.read(&mut chunk) {
|
|
Ok(0) => break,
|
|
Ok(n) => {
|
|
buf.extend_from_slice(&chunk[..n]);
|
|
if buf.windows(4).any(|w| w == b"\r\n\r\n") || buf.len() > 65536 {
|
|
break;
|
|
}
|
|
}
|
|
Err(e) => {
|
|
log(&format!("conn-{id:04} {peer} read failed: {e}"));
|
|
return;
|
|
}
|
|
}
|
|
}
|
|
|
|
// Drain the request body, exactly as the oracle does. This is NOT cosmetic:
|
|
// answering and closing while the client is still sending leaves unread data
|
|
// in the receive queue, and Linux turns that close into an RST rather than a
|
|
// FIN. The oracle has always drained; the first version of this host stopped
|
|
// at the header terminator, which is a behaviour difference invisible to any
|
|
// byte-comparison of the response.
|
|
let body_read = drain_body(&mut tls, &mut buf);
|
|
|
|
let head = String::from_utf8_lossy(&buf);
|
|
let line0 = head.lines().next().unwrap_or("").to_string();
|
|
log(&format!(
|
|
"conn-{id:04} {peer} REQ {line0} req_bytes={} body={}",
|
|
buf.len(),
|
|
match body_read {
|
|
BodyRead::None => "none".to_string(),
|
|
BodyRead::Complete(n) => format!("{n}B complete"),
|
|
BodyRead::Short { got, want } => format!("{got}B of {want}B SHORT"),
|
|
}
|
|
));
|
|
if !line0.is_empty() && !redirector::is_get_server_instance(&line0) {
|
|
log(&format!(
|
|
"conn-{id:04} {peer} NOTE: unexpected request line; answering anyway (oracle behaviour)"
|
|
));
|
|
}
|
|
|
|
let response = redirector::redirect_response(&cfg.adapter);
|
|
if let Err(e) = tls.write_all(&response) {
|
|
log(&format!("conn-{id:04} {peer} write failed: {e}"));
|
|
return;
|
|
}
|
|
let _ = tls.flush();
|
|
log(&format!(
|
|
"conn-{id:04} {peer} SENT {}B serverinstanceinfo -> {}:{}",
|
|
response.len(),
|
|
cfg.adapter.endpoints.advertise,
|
|
cfg.adapter.endpoints.blaze_port
|
|
));
|
|
|
|
// The oracle holds the connection open before closing. That dwell is the
|
|
// only other measured difference between the two implementations, and a
|
|
// redirector that closes at 0ms is not the behaviour FIFA 17 was proven
|
|
// against — so it is reproduced rather than assumed harmless.
|
|
record(
|
|
outcomes,
|
|
ConnOutcome {
|
|
id,
|
|
request_bytes: buf.len(),
|
|
body: body_read,
|
|
response_bytes: response.len(),
|
|
},
|
|
);
|
|
|
|
std::thread::sleep(cfg.close_dwell);
|
|
let _ = tls.shutdown();
|
|
}
|
|
|
|
/// What happened when the request body was drained. Reported per connection
|
|
/// because "the client sent a body we never read" is exactly the class of
|
|
/// divergence a response-bytes comparison cannot see.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub enum BodyRead {
|
|
/// No `Content-Length` header — nothing to drain.
|
|
None,
|
|
Complete(usize),
|
|
Short { got: usize, want: usize },
|
|
}
|
|
|
|
/// Parse `Content-Length` and read the remainder of the body into `buf`.
|
|
///
|
|
/// Mirrors the oracle: header lookup is case-insensitive, and a client that
|
|
/// stops early ends the read rather than hanging until the timeout.
|
|
fn drain_body<S: Read>(stream: &mut S, buf: &mut Vec<u8>) -> BodyRead {
|
|
let Some(sep) = buf.windows(4).position(|w| w == b"\r\n\r\n") else {
|
|
return BodyRead::None;
|
|
};
|
|
let head = String::from_utf8_lossy(&buf[..sep]).to_string();
|
|
let Some(want) = head
|
|
.lines()
|
|
.find(|l| l.to_ascii_lowercase().starts_with("content-length"))
|
|
.and_then(|l| l.split(':').nth(1))
|
|
.and_then(|v| v.trim().parse::<usize>().ok())
|
|
else {
|
|
return BodyRead::None;
|
|
};
|
|
|
|
let mut got = buf.len() - (sep + 4);
|
|
let mut chunk = [0u8; 4096];
|
|
while got < want {
|
|
match stream.read(&mut chunk) {
|
|
Ok(0) | Err(_) => return BodyRead::Short { got, want },
|
|
Ok(n) => {
|
|
buf.extend_from_slice(&chunk[..n]);
|
|
got += n;
|
|
}
|
|
}
|
|
}
|
|
BodyRead::Complete(got)
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn banner_names_the_linked_openssl_and_what_is_advertised() {
|
|
let cfg = RedirectorConfig::for_test("198.51.100.7");
|
|
let b = banner(&cfg);
|
|
assert!(b.contains("openssl="), "{b}");
|
|
assert!(b.contains("commit="), "{b}");
|
|
assert!(b.contains("tls_min="), "{b}");
|
|
assert!(b.contains("198.51.100.7"), "{b}");
|
|
// The cipher list is part of the identity of a compatibility host.
|
|
assert!(b.contains("AES256-GCM-SHA384"), "{b}");
|
|
}
|
|
}
|