Automatic Testing of API Documentation
A presentation at Nordic APIs Platform Summit in in Stockholm, Sweden by Rouven Weßling
A u t o m a t i c
t e s t i n g
o f
( R E S T f u l ) A P I
d o c u m e n t a t i o n N o r d i c
A P I s
P l a t f o r m
S u m m i t ,
O c t o b e r
2 0 1 6 B y
R o u v e n
W e ß l i n g
( ) E c o s y s t e m
D e v e l o p e r
/
D e v e l o p e r
E v a n g e l i s t ,
C o n t e n t f u l @ R o u v e n W e s s l i n g p h o t o
c r e d i t :
b y
S t o c k h o l m ,
A p p r o a c h i n g
B l u e
H o u r T o b i a s
L i n d m a n ( C C
B Y )
A
c o n t e n t
m a n a g e m e n t
d e v e l o p e r
p l a t f o r m
w i t h
a n
A P I
a t
i t s
c o r e .
W h a t
d o
y o u
d o ?
P o w e r e d
b y
D o c u m e n t a t i o n
w i l l
b e
w r o n g
U s e r s
h a t e
t h a t
I t ' s
a l l
i n
t h e
s p e c
a p i
b l u e p r i n t A
p o w e r f u l
h i g h
l e v e l
A P I
d e s c r i p t i o n
l a n g u a g e
f o r
w e b
A P I s .
Q u e s t i o n s
C o l l e c t i o n
[ / q u e s t i o n s ]
C r e a t e
a
N e w
Q u e s t i o n
[ P O S T ] Y o u
m a y
c r e a t e
y o u r
o w n
q u e s t i o n
u s i n g
t h i s
a c t i o n .
I t
t a k e s
a
J S O N o b j e c t
c o n t a i n i n g
a
q u e s t i o n
a n d
a
c o l l e c t i o n
o f
a n s w e r s
i n
t h e f o r m
o f
c h o i c e s . +
R e q u e s t
( a p p l i c a t i o n / j s o n )
{
" q u e s t i o n " :
" F a v o u r i t e
p r o g r a m m i n g
l a n g u a g e ? " ,
" c h o i c e s " :
[
" S w i f t " ,
" P y t h o n " ,
" O b j e c t i v e
C " ,
" R u b y "
]
} +
R e s p o n s e
2 0 1
( a p p l i c a t i o n / j s o n )
H e a d e r s
L o c a t i o n :
/ q u e s t i o n s / 2
B o d y
{
" q u e s t i o n " :
" F a v o u r i t e
p r o g r a m m i n g
l a n g u a g e ? " ,
" p u b l i s h e d _ a t " :
" 2 0 1 5
0 8
0 5 T 0 8 : 4 0 : 5 1 . 6 2 0 Z " ,
" c h o i c e s " :
[
{
" c h o i c e " :
" S w i f t " ,
L e t ' s
g e t
t e s t i n g D R E D D N o
m o r e
o u t d a t e d
A P I
d o c u m e n t a t i o n .
T e s t i n g
r e a d
o n l y n o d e _ m o d u l e s / . b i n / d r e d d
c m a . a p i b
h t t p s : / / a p i . c o n t e n t f u l . c o m
\
m
G E T
H o o k s b e f o r e A l l
c a l l e d
a t
t h e
b e g i n n i n g
o f
t h e
w h o l e
t e s t
r u n b e f o r e E a c h
c a l l e d
b e f o r e
e a c h
H T T P
t r a n s a c t i o n b e f o r e
c a l l e d
b e f o r e
s o m e
s p e c i f i c
H T T P
t r a n s a c t i o n b e f o r e E a c h V a l i d a t i o n
c a l l e d
b e f o r e
e a c h
H T T P
t r a n s a c t i o n
i s
v a l i d a t e d b e f o r e V a l i d a t i o n
c a l l e d
b e f o r e
s o m e
s p e c i f i c
H T T P
t r a n s a c t i o n
i s
v a l i d a t e d a f t e r
c a l l e d
a � e r
s o m e
s p e c i f i c
H T T P
t r a n s a c t i o n
r e g a r d l e s s
i t s
r e s u l t a f t e r E a c h
c a l l e d
a � e r
e a c h
H T T P
t r a n s a c t i o n a f t e r A l l
c a l l e d
a � e r
w h o l e
t e s t
r u n
H o o k s n o d e _ m o d u l e s / . b i n / d r e d d
c m a . a p i b
h t t p s : / / a p i . c o n t e n t f u l . c o m
\
h o o k f i l e s
. / t e s t
h o o k s . j s
\
m
G E T
S k i p p i n g
T e s t s v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; h o o k s . b e f o r e (
" W e b h o o k
c a l l s
W e b h o o k
c a l l
d e t a i l s
G e t
t h e
w e b h o o k
c a l l
d e t a i l s " ,
f u n c t i o n
( t r a n s a c t i o n )
{
t r a n s a c t i o n . s k i p
=
t r u e ;
} ) ;
M u t a t i n g
d a t a v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; h o o k s . b e f o r e (
" E n t r i e s
D e l e t e
a n
E n t r y " ,
f u n c t i o n
( t r a n s a c t i o n )
{
c l i e n t . c r e a t e E n t r y ( e n t r y ,
" 1 2 3 4 " )
} ) ;
R a t e
l i m i t i n g v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; c o n s t
D E L A Y
=
1 0 0 0 / 6 . 0 ; h o o k s . a f t e r E a c h ( f u n c t i o n ( t r a n s a c t i o n ,
d o n e )
{
s e t T i m e o u t ( d o n e ,
D E L A Y ) ; } ) ;
C h a n g i n g
r e q u e s t
d a t a v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; h o o k s . b e f o r e E a c h ( f u n c t i o n
( t r a n s a c t i o n ,
d o n e )
{
t r a n s a c t i o n . f u l l P a t h
=
t r a n s a c t i o n . f u l l P a t h . r e p l a c e ( ' f p 9 1 o e l s z i e a ' ,
' g b k x k l v m o l c 1 ' ) ;
d o n e ( ) ; } ) ;
C h a i
a s s e r t i o n s v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; v a r
a s s e r t
=
r e q u i r e ( ' c h a i ' ) . a s s e r t ; h o o k s . a f t e r E a c h ( f u n c t i o n ( t r a n s a c t i o n ,
d o n e )
{
i f
( ! t r a n s a c t i o n . s k i p )
{
a s s e r t . m a t c h ( t r a n s a c t i o n . r e a l . h e a d e r s [ ' x
c o n t e n t f u l
r e q u e s t
i d ' ] ,
/ ^ c o n t e n t
a p i : [ a
z A
Z 0
9 ] { 2 2 } $ /
}
d o n e ( ) ; } ) ;
S e c u r i t y
�
C e n s o r
p r i v a t e
d a t a v a r
h o o k s
=
r e q u i r e ( ' h o o k s ' ) ; c o n s t
S E C R E T _ H E A D E R S
=
[ ' A u t h o r i z a t i o n ' ] . m a p ( h e a d e r
=
h e a d e r . t o L o w e r C a s e ( ) ) ; h o o k s . a f t e r E a c h V a l i d a t i o n ( f u n c t i o n ( t r a n s a c t i o n ,
d o n e )
{
O b j e c t . k e y s ( t r a n s a c t i o n . r e q u e s t . h e a d e r s ) . f o r E a c h ( f u n c t i o n ( k e y )
{
i f
( S E C R E T _ H E A D E R S . i n d e x O f ( k e y . t o L o w e r C a s e ( ) )
1 )
{
t r a n s a c t i o n . r e q u e s t . h e a d e r s [ k e y ]
=
" * * * H I D D E N
S E C R E T
D A T A * * * " ;
}
} ) ;
d o n e ( ) ; } ) ;
C I
t h e
b e a s t
C o n c l u s i o n B a s e
y o u r
d o c u m e n t a t i o n
o n
a n
A P I
s p e c T e s t
t h a t
s p e c M a k e
i t
p a r t
o f
y o u r
C I D o n ' t
s u b s t i t u t e
y o u r
i n t e g r a t i o n
t e s t s
S l i d e s
a v a i l a b l e
o n
S l i d e s h a r e :
h t t p : / / w w w . s l i d e s h a r e . n e t / r w e s s l i n g / n o r d i c
a p i s
a u t o m a t i c
t e s t i n g
o f
r e s t f u l
a p i
d o c u m e n t a t i o n F o l l o w
m e
o n
T w i t t e r :
@ R o u v e n W e s s l i n g
There are few things more frustrating than documentation that doesn’t match the implementation – especially when it’s documentation for your customers. In actively developed software that is a reoccurring problem. As the product evolves the documentation often lags behind and sometimes breaking changes sneak it, breaking the very examples that are part of the documentation. To avoid all these problems, it’s time to start testing your docs. In this talk I’ll show you what’s currently possible when it comes to testing documentation, how to start testing API documentation written using API Blueprints and how to incorporate it into your workflow.
Video
Buzz and feedback
Here’s what was said about this presentation on Twitter.
