In 2008, Stanley Williams and his group realized memristor in the form of a device in laboratory HP. To create memristor, they used a thin titanium oxide film (TiO₂) [5D.B. Strukov, G.S. Snider, D.R. Stewart, and R.S. Williams, "The missing memristor found", Nature, vol. 453, no. 7191, pp. 80-83.
[http://dx.doi.org/10.1038/nature06932] [PMID: 18451858] ]. The film is connected with 2 poles made of platinum (Pt). One pole is injected with oxygen holes. These oxygen holes are positively charged ions. Therefore, there is a transition layer of TiO₂ with a mixed side and one side is not mixed.

Let us define the thickness of the doped region w, D is the thickness of two layers TiO₂ and R_ON, R_OFF are low and high impedance respectively. Accordingly, the memristance M(t) of the TiO₂ memristor model is given by equation (1)

(1)

Where is the state variable of memristor.

The relationship between the voltage v(t) and current i(t) of the memristor is given in equation (2)

(2)

State variables x(t) are determined:

(3)

Where µ_v is the dopant mobility. Velocity of the width change is linearly proportional to the current as in equation (3). Thus, we call this model linear model.

Various types of nonlinear memristor model have been published. One of them is in the window model in which the state equation is multiplied by window function F_p(w), namely in equation (4)

(4)

Where p is a parameter and F_p(w) is defined by: in equation (5)

(5)

This is called the nonlinear drift model. It means that as the p decreases, the nonlinearity increases. In other words, as the p increases, the model tends to the linear one as shown in Fig. (1).

Fig. (1) Memristance of memristor with p parameter.

The conventional memristor bridge synapse operates like a Wheatstone bridge consisting of four identical memristors with the polarities indicated in Fig. (2a) [10C. Zamarreño-Ramos, L.A. Camuñas-Mesa, J.A. Pérez-Carrasco, T. Masquelier, T. Serrano-Gotarredona, and B. Linares-Barranco, "On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex", Front. Neurosci., vol. 5, p. 26.
[http://dx.doi.org/10.3389/fnins.2011.00026] [PMID: 21442012] ]. The input signal is applied from the left side of the circuit and the output is taken from two middle nodes as a differential form. When strong pulse Vin is applied at the input, the memristance of each memristor is increased or decreased depending upon its polarity. For instance, when a positive pulse is applied as an input, the memristances of M1 and M4 whose polarities are forward biased will decrease. On the other hand, the memristances of M2 and M3 whose polarities are reverse biased, will increase. It follows that the voltage V_A at node A with respect to the ground becomes smaller than the voltage V_B at node B. Since, the node voltage V_A is less than V_B, the output voltage Vout across the bridge is negative weight. Memristor bridge circuit linearly generates synaptic weights in the range [-1;1] [10C. Zamarreño-Ramos, L.A. Camuñas-Mesa, J.A. Pérez-Carrasco, T. Masquelier, T. Serrano-Gotarredona, and B. Linares-Barranco, "On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex", Front. Neurosci., vol. 5, p. 26.
[http://dx.doi.org/10.3389/fnins.2011.00026] [PMID: 21442012] ].

Fig. (2) (a) Memristor bridge circuit [10C. Zamarreño-Ramos, L.A. Camuñas-Mesa, J.A. Pérez-Carrasco, T. Masquelier, T. Serrano-Gotarredona, and B. Linares-Barranco, "On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex", Front. Neurosci., vol. 5, p. 26. [http://dx.doi.org/10.3389/fnins.2011.00026] [PMID: 21442012] ].(b)The proposed scheme of synapse based on two successive memristors.

Two successive memristors are similar to half-bridge circuit consisting of two memristors shown in Fig. (2b). When we want to get positive or negative value, strong pulse Vin is applied at the input, the memristance of each memristor is increased or decreased depending upon its polarity. For instance, when a positive pulse is applied as an input, the memristances of M1 decrease. On the other hand, the memristances of M2 increase. Because the circuit acts as a half-bridge, we can only program positive weights in the range [0;1].

If Vin is input signal applied to the memristor haft-bridge circuit in Fig. (2b) at time t, the input voltage will be divided by the well-known “voltage – divider formula” as follows: in eq 6 and equation 7

(6)

(7)

Where M1, and M2 denote the corresponding memristances of the memristors at time t, in Fig. (2b).

The output voltage Vout of the circuit is equal to the voltage at terminal A, namely in equation 8

(8)

From (8), the voltage Vout is given in equation 9,

(9)

Where represents the synaptic weight of the successive memristor circuit.

The successive memristor circuit could only generate positive weights in the range [0;1] corresponding to each weight of memristor circuit. Thus, we have to map weights from the range [0;1] to range [-1;1] so that we can program the synapse weighting.

Fig. (3a) describes a perceptron network with m inputs and m synapses. We have the following equation: (10)

Fig. (3) The synapse circuit consists of two successive memristors. (a) Setting positive weightings in range [0;1] to achieve Vplus (b) Vminus scheme to compensate the weightings in range [-1,1] (c) Output value, outk.

(10)

Conventionally, weights of w₁, w₂, ..., w_m covering in [-1,1] are used to set synapse weighting in bridge memristor synapse [10C. Zamarreño-Ramos, L.A. Camuñas-Mesa, J.A. Pérez-Carrasco, T. Masquelier, T. Serrano-Gotarredona, and B. Linares-Barranco, "On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex", Front. Neurosci., vol. 5, p. 26.
[http://dx.doi.org/10.3389/fnins.2011.00026] [PMID: 21442012] ]. The input signals are from in₁ to in_m. Y is an output summation of the perceptron network. However, the circuit of two successive memristors only creates weighting in range [0;1] as explained in equation (8). To transfer the weights w₁, w₂, ..., w_m from the range [-1;1] to the new range [0;1], this weight is divided by 2, then adding 0.5. For instance, If the w₁ is -1 weighting in range [-1;1], the new w₁ weighting will be 0 in range [0,1]. In other hand, If w₁ weighting is 1, the new w₁ weighting will be 1 in range [0;1].

From (10), after transferring weights to the new range [0;1], we have the following equation: (11)

(11)

Where, Z is new output when in range [0;1].

From (11), we have the following equation: (12)

(12)

From (10) and (12), we get this equation: (13)

(13)

We can see the formula (13) shown in Fig. (3c). Where, Z is V_plus, A is V_minus and Y is V_out.

In Fig. (3a), the positive weights are installed in the range [0;1]. In Fig. (3b), negative values are created to compensate weightings in the range [-1;1]. In Fig. (3c), we sum positive and negative values and double the result to achieve V_out . Then V_out is compared with V_ref to determine output of out_k with k corresponding to the number of output neurons in equation (14).

(14)

The parity function is used very often in digital systems for error detection and correction. Parity system is also used to detect hardware failures in digital memory. The digital additions and multiplications also require parity circuits. The parity circuit is usually implemented by cascading XOR gates. This solution for N-bit parity requires N-1 XOR gates. Thus, this circuit consists of several layers and introduces significant delays. The modular neural networks are suitable to solve these parity problems [11S. Sayyaparaju, S. Amer, and G.S. Rose, "A bi-memristor synapse with spike-timing-dependent plasticity for on-chip learning in memristive neuromorphic systems", 19^th Int Sympo Quality Electron Design (ISQED), .
[http://dx.doi.org/10.1109/ISQED.2018.8357267] ].

Fig. (5) shows a detail implementation circuit of two successive memristor architecture. A set of x₁,...,x_n plays a role as inputs of neural network system. A two successive memristor array represents a weight array of Ψ₁,...,Ψ_n. A negative amplifier with n inputs are connected to n resistors. To get value -½ as expected, we have: in equation 15

(15)

if we assume that , we can obtain in equation (16)

(16)

Next, a multiplier of 2 with two inputs is presented. The output of V_out is calculated by in equation 17

(17)

With ,

(18)

Fig. (5) Detail circuit of successive memristor architecture.

Fig. (6) shows the neural network model of training 3-bit parity. The 3-bit parity is trained by a neural network with three inputs, six neurons in hidden layer and one output. The weights of hidden layer are a set of and the weights of output layer are a set of . Beside, the biases are added in each neuron in hidden layer and output layer. The biases of hidden layer are a set of and the bias of the output layer is b₂. The input is in the range of [-1; 1] and the output is in the range of [-1; 1].

Fig. (6) The neural network model to train 3-bit parity problem with three inputs, six neurons in hidden layer and one output.

Table 1 shows the truth table of 3-bit parity problem and the encoding used for input bits and output bits for implementations to satisfy the weight programming process. The odd parity problem counts the number of bit 1s to determine the output value. If the number of bit 1s is odd, the output bit will be 1. If the number of bit 1s is even, the output bit will be 0. The bit 0 will be encoded to -1, the bit 1 will be encoded to 1.

Table 1
Truth table of odd parity and the encoding used for input bits and output bits.

Fig. (7) shows detail connections of 3-bit parity training circuit. Each neuron in the hidden layer is performed by each column in the circuit. It includes 4 pairs of memristors consisting of 3 weights of and 1 weight of bias input. For example, the first column has 3 pairs of memristors that are and fourth pair is vb_1,1. The output of the first column is Z₁. Similarly, the outputs of other six columns are a set from Z₁ to Z₆. After that, all neurons in hidden layer are connected to the output neuron. It is performed by orange column. It includes 7 pairs of memristors that are 6 weights of the output layer from w₁ to w₆ and one weight of the input bias as vb₂. A comparison circuit with reference voltage is added to generate the output in range [-1; 1] as Table 1. All resistor values are explained in equations from 15 to 18.

Fig. (7) Detail connection of the 3-bit parity circuit.

The Back-propagation algorithm is utilized to updating memristance. The training algorithm based on the back-propagation is stated as:

1. Set random weight for each pair of memristors.
2. For each input x:
3. Apply input to determine hidden layer and output values.
4. Calculate error between the neuron output and desired output: d-y.
5. Update weights of output layer and hidden layer.
6. If the error is not covered, go to step 2, else, finish training.

(19)

(20)

The operations in the training process for the neural network can be broken into three major steps as the following:

1. Apply inputs to hidden layer and output layer to determine neuron errors.
2. Back-propagate the errors through the output layer weights and hidden layer weights.
3. Update the synaptic weights for both layers of neurons.

Step 1: A set of inputs is applied to the hidden layer, and the output layer is measured. Then, the algorithm determines the difference between neuron output and the desired output. These values are generated using a comparator that provides a discretized error value of +1 or -1. Thus, these errors can easily be recorded in binary form for later use.

Step 2: The errors are applied to the output layer weights and hidden layer weights as Eq. 19 and Eq. 20

Step 3: To update weight, based on the pulse difference between reference pulse and input pulse, we get plus pulse width modulation circuit as shown in waveform of Fig. (8). As known, the memristance is changed when applying positive pulse or negative pulse. Because memristor can store its state, we just need to update ∆_w or ∆_v instead of w_k+1 or v_k+1 as shown in Fig. (9).

Fig. (8) Waveform shows different pulse widths for different input magnitude. Vout shows pulse width of modulation circuit output.

Fig. (9) Circuit model of the neural network.

Fig. (10) shows the training curve obtained from simulations of MATLAB – CADENCE SPECTRE utilizing the back-propagation algorithm. After nine epochs, the error approaches to zero, it means that the training process was successful.

Fig (10) Learning curve for three bit input parity problem.

Fig. (11) shows the prototype of the five digits. Each digit has 10 samples and is put into training process. The more the sample, the higher the training accuracy. Here we only train 10 samples with each digit. To increase the precision of the actual recognition application, the more samples should be put into training.

Fig. (11) Training images.

Each sample in Fig. (11) is a 5x4 sized character and each weight will be programmed by synaptic circuit including two successive memristors. Thus, we design 20 memristor synaptic circuits for 20 synapse weights.

Fig. (12) shows the results of simulation output voltage of each digit from 1 to 5. The results show that we can identify the output digit based on the output voltage level. Among five output voltages, the digit can be recognized by setting a threshold voltage level. For instance, when we use digit 1 recognition circuit that we will take the result as in Fig. (12a). The output voltage is 1 V if input is digit 1 and output is 0 if the inputs are the other digits such as 2, 3, 4 or 5. Similarly, the output voltage of digit-2 recognition circuit is 1V as in Fig. (12b) if input is digit 2, else output voltage is 0V in the other digit inputs. Figs. (12c, d, e) is corresponding to digit 3, digit 4, and digit 5, respectively.

Fig. (12) Output simulation results for 5 digits with synapse based on two successive memristors.

Fig. (13) describes the character in interference noise. We take digit-1 character as an example. The percentage of noise is shown in this figure. Noise ratio is calculated by noise bits over a total data bits. For example, a noisy bit is added to a 5 x 4 size sample, we achieve a noise rate of 5%. Table 2 shows the recognition rate when the interference noise is occurring. Recognition accuracy will greatly reduce according to the number of the added noise bit.

Fig. (13) Noisy patterns added for recognition test at 5%, 10%, and 15%.

Table 2 shows the successful identification rate for 5 digits. The results show the same identification rate among the two circuit schemes. There is no difference in recognition rate between the two schemes. When noise patterns increase, the recognition rate will greatly decrease. For example, the digit 1 gets 100% successful identification rate with no noise effect in both the conventional memristor bridge circuit and the proposed successive memristor circuit. At 15% added noise patterns, the recognition rate of 60% is achieved for both techniques of memristor bridge circuit and successive memristor circuit. However, the successive memristor circuit will save half the number of memristors. The CMOS transistors in the proposed technique are also less than in the memristor bridge circuit. Thus, area and power consumption of the proposed circuit will be smaller for the conventional memristor bridge circuit.

Table 2
Recognition rate in various noise patterns.

The system consists of five recognition circuits in parallel; each circuit corresponds to a digit from 1 to 5. The conventional identification circuit uses synapses that are conducted by bridge memristor circuits. Each circuit includes 20 synapses. Each synapse uses 4 memristors. Thus, the system uses 400 memristors in all 5 identification circuits for digits from 1 to 5. The number of CMOS transistor is 62 for each identification circuit. The system consists of 5 circuits consuming 310 CMOS transistors as shown in Table 3.

Table 3
Comparison in the overhead area.

The proposed identification circuit uses 20 synapses that are constructed from two successive memristors. The proposed system has all 5 circuits, so total memristors are 200. We have a total of 140 CMOS corresponding to 5 proposed recognition circuits for 5 digits from 1 to 5 as shown in Table 3.

Thus, the synaptic circuit using two successive memristors occupies a smaller area than the conventional memristor bridge synapse circuit.

Fig. (14) and Table 4 show clearly that the synapse of the two-successive memristors consumes less than the conventional memristor bridge synapse circuit in terms of power dissipation. The proposed technique can save upto 76.88% power loss compared to the conventional memristor bridge circuit in case of No.5 digit recognition. This is because the area of the proposed successive memristor circuit is designed with smaller numbers of memristor and CMOS transistor compared to the conventional memristor bridge circuit. Here, the accuracy recognition rate is the same among two techniques.

Fig. (14) Power consumption in the two character identification circuits for each digit.

Table 4
Power consumption in the character recognition circuits for each number and reduction percentage.