Two-particle correlations in
p
Pb
collisions at a nucleon-nucleon center-of-mass energy of
5.02
TeV
are studied as a function of the pseudorapidity separation (
Δ
η
) of the particle pair at small relative azimuthal angle (
|
Δ
ϕ
|
<
π
/
3
). The correlations are decomposed into a jet component that dominates the short-range correlations (
|
Δ
η
|
<
1
), and a component that persists at large
Δ
η
and may originate from collective behavior of the produced system. The events are classified in terms of the multiplicity of the produced particles. Finite azimuthal anisotropies are observed in high-multiplicity events. The second and third Fourier components of the particle-pair azimuthal correlations,
V
2
and
V
3
, are extracted after subtraction of the jet component. The single-particle anisotropy parameters
v
2
and
v
3
are normalized by their laboratory frame midrapidity value and are studied as a function of
η
c.m.
. The normalized
v
2
distribution is found to be asymmetric about
η
c.m.
=
0
, with smaller values observed at forward pseudorapidity, corresponding to the direction of the proton beam, while no significant pseudorapidity dependence is observed for the normalized
v
3
distribution within the statistical uncertainties.