Neural Networks Without Magic: Occam’s Razor and a Programmer’s View

Artificial intelligence doesn’t have to be esoteric. High school math, a few derivatives, and clean code in C++/MQL5 are enough. No magic—just numbers in matrices, functions, and discipline.


A Problem Without Fog: What We Really Need

Texts about neural networks often contain a lot of “smoke and mirrors.” I used Occam’s razor and kept only what is necessary—the mathematical minimum everyone remembers from high school:

  • multiplication and addition of numbers,
  • simple nonlinear functions (tanh, sigmoid, ReLU),
  • basic work with vectors and matrices,
  • and derivatives for learning (backpropagation).

That’s it. Everything else is just comfort, not necessity.


Derivatives: The Only “Optional Mandatory” Chapter

Without derivatives, we only get the result of the forward pass. But learning requires knowing how to adjust each weight. This is where the chain rule comes in: compute the error, send it back through the layers, and adjust weights and biases according to the derivatives of activations and linear parts.

  • Error: difference between prediction and reality (MSE).
  • Gradient: derivative of the composite function (chain rule).
  • Update: a small step in the direction of error reduction (SGD).

If you can handle the derivative of sigmoid, tanh, and “ReLU derivative,” you’ve covered 99% of what’s needed.


C++ and MQL5: Math Translated into Practice

In C++ I have a lightweight DLL: dense layers, activations, forward pass, backpropagation, and single-sample training (SGD). Everything runs through a simple C API (handle, no global singletons). In MQL5, the EA takes data directly from the chart, normalizes it, feeds it into the network, and draws outputs.

  • y = f(Wx + b) — forward pass.
  • dL/dW, dL/db — gradients via activation derivatives.
  • SGD, MSE, gradient clipping, He/Xavier init.

Result: AI stops looking magical. What remains is programming craft with understandable behavior.


Occam’s Razor in Bullet Points

  • Forward pass: matrix–vector multiplication.
  • Activation: tanh/sigmoid/ReLU as ordinary functions.
  • Loss (MSE): mean squared error.
  • Backpropagation: pure chain rule.

Complexity arises from stacking simple things, not from mystery.


What This Path Gave Me

I see the network as a system of calculation, not a prophecy. I understand both its limits and possibilities. And I have a tool I can expand: save/load weights, add Adam, more outputs, connect it to trading logic.

High school math, patience, and the will to remove ballast are enough.


Conclusion

Neural networks are not a black box. They are numbers in matrices and a few derivatives—a mathematical mechanism translated into the languages I like: C++ and MQL5.


Download (EA / sources)

NNMQL5 Documentation:


A lightweight DLL exposing a C API for a simple MLP (stack of dense layers) with:

  • forward inference (NN_Forward)
  • online single-sample training (NN_TrainOne, SGD)
  • multi-network instance management via integer handles

The network is stateful (weights in RAM), no persistence.

Features

  • Dense layers: W[out x in], bias b[out]
  • Activations: SIGMOID, RELU, TANH, LINEAR, SYM_SIG (symmetric sigmoid)
  • Weight init: He for ReLU, Xavier-like for others
  • Basic gradient clipping (per-neuron, ±5) in backprop
  • Double precision, x64 build (MSVC)
  • C ABI (no name mangling) → easy to call from MQL5

Limitations (by design)

  • SGD single sample only (no mini-batches, no Adam, etc.)
  • No model save/load (serialization) yet
  • No RNG seeding API (see “Reproducibility”)

Binary / Build Info

  • Platform: Windows x64, MSVC (Visual Studio)
  • Exports: extern "C" __declspec(dllexport)
  • Exceptions: never cross the C boundary; API returns bool/int
  • Threading: instance table guarded by std::mutex. Per-network ops are not re-entrant → serialize calls per handle (MT5 EAs are typically single-threaded)

Suggested VS settings

  • Configuration: Release | x64
  • C++: /std:c++17 (or newer), /O2, /EHsc
  • Runtime: /MD (default; shared CRT)

API (C interface)

All functions use C ABI. Signatures:

// Create a new (empty) network; returns positive handle or 0 on failure.
int  NN_Create(void);

// Free network by handle (idempotent).
void NN_Free(int h);

// Add a dense layer to network h.
// act: 0=SIGMOID, 1=RELU, 2=TANH, 3=LINEAR, 4=SYM_SIG
bool NN_AddDense(int h, int inSz, int outSz, int act);

// Declared input/output sizes (0 if handle invalid or network empty).
int  NN_InputSize(int h);
int  NN_OutputSize(int h);

// Forward pass: in[in_len] → out[out_len]. False if sizes/handle invalid.
bool NN_Forward(int h, const double* in, int in_len,
                double* out, int out_len);

// One SGD training step with MSE.
// If mse != nullptr, stores current MSE (after update).
bool NN_TrainOne(int h, const double* in, int in_len,
                 const double* tgt, int tgt_len,
                 double lr, double* mse);

Activation codes (act)

CodeActivationNotes
0SIGMOID1/(1+e−x)1/(1+e^{-x})1/(1+e−x)
1RELUmax⁡(0,x)\max(0,x)max(0,x), He init
2TANHtanh⁡(x)\tanh(x)tanh(x)
3LINEARIdentity
4SYM_SIG2σ(x)−1∈(−1,1)2\sigma(x)-1 \in (-1,1)2σ(x)−1∈(−1,1)

Lifecycle (recommended)

  1. h = NN_Create()
  2. Topology via NN_AddDense(...) in order
    • First layer sets input size
    • Last layer sets output size
    • Sizes must match: out(k-1) == in(k) → returns false if not
  3. Optionally check NN_InputSize(h), NN_OutputSize(h)
  4. Training: loop NN_TrainOne(h, in, in_len, tgt, tgt_len, lr, &mse)
  5. Inference: NN_Forward(h, in, in_len, out, out_len)
  6. NN_Free(h)

Semantics

  • Validation:NN_Forward/NN_TrainOne return false if:
    • invalid handle
    • network has no layers
    • buffer lengths don’t match topology
  • Memory safety: the DLL never owns your buffers; you provide valid double* memory
  • Numerics:
    • I/O and targets are double
    • Gradient clipping ±5 per neuron
    • He/Xavier-like init as described
  • Multiple instances: any number of networks (separate handles)
  • Re-entrancy: don’t call into the same handle concurrently

Reproducibility (RNG)

  • Weights use std::rand() with U[-1,1]*scale
  • Deterministic runs: seed std::srand(...)in the same CRT context as the DLL
    • With /MD (shared CRT) seeding in host may affect DLL RNG — not strictly guaranteed
    • Clean solution: add an export like NN_Seed(unsigned) (not included in this build)

MQL5 usage

Import block

#import "NNMQL5.dll"
int  NN_Create();
void NN_Free(int h);
bool NN_AddDense(int h,int inSz,int outSz,int act);
bool NN_Forward(int h,const double &in[],int in_len,double &out[],int out_len);
bool NN_TrainOne(int h,const double &in[],int in_len,const double &tgt[],int tgt_len,double lr,double &mse);
int  NN_InputSize(int h);
int  NN_OutputSize(int h);
#import

Note: In MQL5, arrays are passed as const double &arr[].

Minimal network + inference (e.g., 4→8→1)

int h = NN_Create();
if(h<=0){ Print("Create failed"); return; }

bool ok = true;
ok &= NN_AddDense(h, 4, 8, 1); // RELU
ok &= NN_AddDense(h, 8, 1, 3); // LINEAR
if(!ok){ Print("Topology failed"); NN_Free(h); return; }

double x[4] = {1,2,3,4};
double y[1];
if(!NN_Forward(h, x, 4, y, 1)) Print("Forward failed");
Print("y=", y[0]);

NN_Free(h);

One SGD step

double x[4] = { /* features */ };
double t[1] = { /* target   */ };
double mse=0.0, lr=0.01;

bool ok = NN_TrainOne(h, x, 4, t, 1, lr, mse);
if(!ok) Print("TrainOne failed");
else    Print("MSE=", DoubleToString(mse,6));

Typical EA flow (pseudo)

  • OnInit: create network + topology
  • OnTimer: sample from dataset → NN_TrainOne(...), log MSE, stop when target reached
  • OnTick/OnCalculate: NN_Forward(...) for predictions

Practical tips

  • Normalize inputs/targets (mean/std or min-max) for stable learning
  • Tune lr: typically higher for ReLU, lower for “soft” activations
  • Prefer LINEAR output for regression targets
  • For time series: avoid look-ahead leak; train only from past to future
  • Adaptive LR (plateau/cosine/CLR) can be implemented in MQL; lr is free parameter

Quick reference (return values)

FunctionSuccess (true/>0)False/0 means
NN_CreatePositive handleAllocation failed (rare)
NN_Free— (idempotent)
NN_AddDenseLayer addedInvalid handle / size mismatch
NN_InputSizeDeclared input size0 (invalid handle / empty network)
NN_OutputSizeDeclared output size0 (invalid handle / empty network)
NN_Forwardout[] filledInvalid handle/sizes/empty network
NN_TrainOnePerformed update, *mse if providedInvalid handle/sizes/empty network

Implementation notes (for audit)

  • Forward: per layer z=Wx+bz = W x + bz=Wx+b; caches last_in, last_z, last_out
  • Backward: dL/dz = dL/dy ⊙ f'(z) (ReLU uses x>0); clip ±5;
    update b -= lr * dL/dz, W -= lr * (dL/dz) * last_in^T
  • Loss: MSE = mean((y - t)^2), gradient dL/dy = 2(y - t)/n
  • Init: u ∈ U[-1,1], w = u * scale, scale = sqrt(2/in) for ReLU, else sqrt(1/in)

Performance notes

  • Don’t create/destroy networks every tick; allocate in OnInit, train via timer
  • Reuse MQL arrays to avoid repeated allocations
  • Balance in_len (lookback) vs. network width/depth for throughput

MQL5 compatibility

  • double ABI matches (8B)
  • Arrays passed as references in MQL5 (const double &arr[]); the DLL reads/writes via raw pointers; you guarantee length
  • bool mapping is standard (0/1)

Troubleshooting

  • NN_AddDense returns false: size mismatch (e.g., 8→16 then 32→1). Fix to 8→16→1 or adjust layer sizes.
  • NN_Forward/TrainOne false: wrong in_len/out_len/tgt_len or invalid handle / empty network.
  • “No learning”: try different lr, normalize data, confirm target range matches activation (e.g., don’t use sigmoid for unbounded regression).

License note

“MIT-like spirit”: use freely, please keep attribution.


Mini Quick-Start (complete MQL5 skeleton)

#import "NNMQL5.dll"
int  NN_Create(); void NN_Free(int h);
bool NN_AddDense(int h,int inSz,int outSz,int act);
bool NN_Forward(int h,const double &in[],int in_len,double &out[],int out_len);
bool NN_TrainOne(int h,const double &in[],int in_len,const double &tgt[],int tgt_len,double lr,double &mse);
int  NN_InputSize(int h); int NN_OutputSize(int h);
#import

int h=-1;
int OnInit(){
  h=NN_Create(); if(h<=0) return INIT_FAILED;
  if(!NN_AddDense(h, 32, 64, 1)) return INIT_FAILED; // RELU
  if(!NN_AddDense(h, 64,  1, 3)) return INIT_FAILED; // LINEAR
  return INIT_SUCCEEDED;
}
void OnDeinit(const int r){ if(h>0) NN_Free(h); }

void OnTimer(){
  static double x[32], t[1], y[1]; double mse;
  // ...fill x[] and t[]...
  NN_TrainOne(h, x, 32, t, 1, 0.01, mse);
  NN_Forward(h, x, 32, y, 1);
  Print("y=",y[0]," mse=",mse);
}

MQL5 ↔ DLL function reference

int NN_Create();

What it does: Creates an empty NN instance and returns its handle (>0).
Return: >0 = valid handle, 0 = allocation failed (rare).
Note: The network is empty until you add layers via NN_AddDense.


void NN_Free(int h);

What it does: Frees the network identified by handle h.
Params:

  • h – handle returned by NN_Create.
    Behavior: Idempotent (calling on an invalid/nonexistent handle is harmless). After freeing, h is invalid.

bool NN_AddDense(int h, int inSz, int outSz, int act);

What it does: Adds a dense layer to network h.
Params:

  • h – network handle.
  • inSz – input size of the layer.
  • outSz – number of neurons (outputs) of the layer.
  • act – activation code:
    • 0 SIGMOID, 1 RELU, 2 TANH, 3 LINEAR, 4 SYM_SIG (symmetric sigmoid).
      Return: true if the layer was added; false if the handle is invalid or dimensions don’t match (i.e., previous layer’s outSz ≠ this layer’s inSz).
      Side effects:
  • The first added layer fixes the network input size (NN_InputSize).
  • The last added layer defines the output size (NN_OutputSize).
  • Biases start at zero; weights use He (for ReLU) or Xavier-like init.

bool NN_Forward(int h, const double &in[], int in_len, double &out[], int out_len);

What it does: Inference – forward pass through network h.
Params:

  • h – network handle.
  • in[] – input vector (MQL5 passes arrays as “reference to array”).
  • in_len – length of in[]; must equal NN_InputSize(h).
  • out[] – preallocated output array; DLL writes the result here.
  • out_len – length of out[]; must equal NN_OutputSize(h).
    Return: true = output written to out[]; false = invalid handle, empty network, or size mismatch.
    Note: Does not change weights; double precision.

bool NN_TrainOne(int h, const double &in[], int in_len, const double &tgt[], int tgt_len, double lr, double &mse);

What it does: One SGD training step on a single sample.
Params:

  • h – network handle.
  • in[] – input vector. in_len == NN_InputSize(h).
  • tgt[] – target vector. tgt_len == NN_OutputSize(h).
  • lr – learning rate (control it from EA; normalize data for stability).
  • mseoutput param: MSE of the current forward. Technically: forward → MSE computed → gradient → weights updated.
    Return: true = update applied (weights changed); false = invalid handle/sizes/empty network.
    Note: Includes gradient clipping (±5 per neuron) for stability.

int NN_InputSize(int h);

What it does: Returns the declared input dimension of network h.
Return: >0 = input size; 0 = invalid handle or network has no layers.


int NN_OutputSize(int h);

What it does: Returns the declared output dimension of network h.
Return: >0 = output size; 0 = invalid handle or network has no layers.


Contracts (preconditions)

  • NN_AddDense: outSz(prev) == inSz(new); otherwise false.
  • NN_Forward: in_len == NN_InputSize(h) and out_len == NN_OutputSize(h).
  • NN_TrainOne: in_len == NN_InputSize(h) and tgt_len == NN_OutputSize(h).
  • Arrays in[]/out[]/tgt[] must be correctly allocated by the caller; the DLL only reads/writes.

Author: Tomáš Bělák